]> git.saurik.com Git - bison.git/blame - doc/bison.texinfo
doc: add bibliography to manual.
[bison.git] / doc / bison.texinfo
CommitLineData
bfa74976
RS
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
df1af54c
JT
4@include version.texi
5@settitle Bison @value{VERSION}
bfa74976
RS
6@setchapternewpage odd
7
5378c3e7 8@finalout
5378c3e7 9
13863333 10@c SMALL BOOK version
bfa74976 11@c This edition has been formatted so that you can format and print it in
13863333 12@c the smallbook format.
bfa74976
RS
13@c @smallbook
14
91d2c560
PE
15@c Set following if you want to document %default-prec and %no-default-prec.
16@c This feature is experimental and may change in future Bison versions.
17@c @set defaultprec
18
8c5b881d 19@ifnotinfo
bfa74976
RS
20@syncodeindex fn cp
21@syncodeindex vr cp
22@syncodeindex tp cp
8c5b881d 23@end ifnotinfo
bfa74976
RS
24@ifinfo
25@synindex fn cp
26@synindex vr cp
27@synindex tp cp
28@end ifinfo
29@comment %**end of header
30
fae437e8 31@copying
bd773d73 32
8a4281b9
JD
33This manual (@value{UPDATED}) is for GNU Bison (version
34@value{VERSION}), the GNU parser generator.
fae437e8 35
575619af
JD
36Copyright @copyright{} 1988-1993, 1995, 1998-2011 Free Software
37Foundation, Inc.
fae437e8
AD
38
39@quotation
40Permission is granted to copy, distribute and/or modify this document
8a4281b9 41under the terms of the GNU Free Documentation License,
804e83b2 42Version 1.3 or any later version published by the Free Software
c827f760 43Foundation; with no Invariant Sections, with the Front-Cover texts
8a4281b9 44being ``A GNU Manual,'' and with the Back-Cover Texts as in
c827f760 45(a) below. A copy of the license is included in the section entitled
8a4281b9 46``GNU Free Documentation License.''
c827f760 47
389c8cfd 48(a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
8a4281b9
JD
49modify this GNU manual. Buying copies from the FSF
50supports it in developing GNU and promoting software
389c8cfd 51freedom.''
fae437e8
AD
52@end quotation
53@end copying
54
e62f1a89 55@dircategory Software development
fae437e8 56@direntry
8a4281b9 57* bison: (bison). GNU parser generator (Yacc replacement).
fae437e8 58@end direntry
bfa74976 59
bfa74976
RS
60@titlepage
61@title Bison
c827f760 62@subtitle The Yacc-compatible Parser Generator
df1af54c 63@subtitle @value{UPDATED}, Bison Version @value{VERSION}
bfa74976
RS
64
65@author by Charles Donnelly and Richard Stallman
66
67@page
68@vskip 0pt plus 1filll
fae437e8 69@insertcopying
bfa74976
RS
70@sp 2
71Published by the Free Software Foundation @*
0fb669f9
PE
7251 Franklin Street, Fifth Floor @*
73Boston, MA 02110-1301 USA @*
9ecbd125 74Printed copies are available from the Free Software Foundation.@*
8a4281b9 75ISBN 1-882114-44-2
bfa74976
RS
76@sp 2
77Cover art by Etienne Suvasa.
78@end titlepage
d5796688
JT
79
80@contents
bfa74976 81
342b8b6e
AD
82@ifnottex
83@node Top
84@top Bison
fae437e8 85@insertcopying
342b8b6e 86@end ifnottex
bfa74976
RS
87
88@menu
13863333
AD
89* Introduction::
90* Conditions::
8a4281b9 91* Copying:: The GNU General Public License says
f5f419de 92 how you can copy and share Bison.
bfa74976
RS
93
94Tutorial sections:
f5f419de
DJ
95* Concepts:: Basic concepts for understanding Bison.
96* Examples:: Three simple explained examples of using Bison.
bfa74976
RS
97
98Reference sections:
f5f419de
DJ
99* Grammar File:: Writing Bison declarations and rules.
100* Interface:: C-language interface to the parser function @code{yyparse}.
101* Algorithm:: How the Bison parser works at run-time.
102* Error Recovery:: Writing rules for error recovery.
bfa74976 103* Context Dependency:: What to do if your language syntax is too
f5f419de
DJ
104 messy for Bison to handle straightforwardly.
105* Debugging:: Understanding or debugging Bison parsers.
ff7571c0 106* Invocation:: How to run Bison (to produce the parser implementation).
f5f419de
DJ
107* Other Languages:: Creating C++ and Java parsers.
108* FAQ:: Frequently Asked Questions
109* Table of Symbols:: All the keywords of the Bison language are explained.
110* Glossary:: Basic concepts are explained.
111* Copying This Manual:: License for copying this manual.
5e528941 112* Bibliography:: Publications cited in this manual.
f5f419de 113* Index:: Cross-references to the text.
bfa74976 114
93dd49ab
PE
115@detailmenu
116 --- The Detailed Node Listing ---
bfa74976
RS
117
118The Concepts of Bison
119
f5f419de
DJ
120* Language and Grammar:: Languages and context-free grammars,
121 as mathematical ideas.
122* Grammar in Bison:: How we represent grammars for Bison's sake.
123* Semantic Values:: Each token or syntactic grouping can have
124 a semantic value (the value of an integer,
125 the name of an identifier, etc.).
126* Semantic Actions:: Each rule can have an action containing C code.
127* GLR Parsers:: Writing parsers for general context-free languages.
128* Locations Overview:: Tracking Locations.
129* Bison Parser:: What are Bison's input and output,
130 how is the output used?
131* Stages:: Stages in writing and running Bison grammars.
132* Grammar Layout:: Overall structure of a Bison grammar file.
bfa74976 133
8a4281b9 134Writing GLR Parsers
fa7e68c3 135
8a4281b9
JD
136* Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
137* Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
20be2f92 138* GLR Semantic Actions:: Considerations for semantic values and deferred actions.
ca2a6d15 139* Semantic Predicates:: Controlling a parse with arbitrary computations.
8a4281b9 140* Compiler Requirements:: GLR parsers require a modern C compiler.
fa7e68c3 141
bfa74976
RS
142Examples
143
f5f419de
DJ
144* RPN Calc:: Reverse polish notation calculator;
145 a first example with no operator precedence.
146* Infix Calc:: Infix (algebraic) notation calculator.
147 Operator precedence is introduced.
bfa74976 148* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 149* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
f5f419de
DJ
150* Multi-function Calc:: Calculator with memory and trig functions.
151 It uses multiple data-types for semantic values.
152* Exercises:: Ideas for improving the multi-function calculator.
bfa74976
RS
153
154Reverse Polish Notation Calculator
155
f5f419de
DJ
156* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
157* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
158* Rpcalc Lexer:: The lexical analyzer.
159* Rpcalc Main:: The controlling function.
160* Rpcalc Error:: The error reporting function.
161* Rpcalc Generate:: Running Bison on the grammar file.
162* Rpcalc Compile:: Run the C compiler on the output code.
bfa74976
RS
163
164Grammar Rules for @code{rpcalc}
165
13863333
AD
166* Rpcalc Input::
167* Rpcalc Line::
168* Rpcalc Expr::
bfa74976 169
342b8b6e
AD
170Location Tracking Calculator: @code{ltcalc}
171
f5f419de
DJ
172* Ltcalc Declarations:: Bison and C declarations for ltcalc.
173* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
174* Ltcalc Lexer:: The lexical analyzer.
342b8b6e 175
bfa74976
RS
176Multi-Function Calculator: @code{mfcalc}
177
f5f419de
DJ
178* Mfcalc Declarations:: Bison declarations for multi-function calculator.
179* Mfcalc Rules:: Grammar rules for the calculator.
180* Mfcalc Symbol Table:: Symbol table management subroutines.
bfa74976
RS
181
182Bison Grammar Files
183
184* Grammar Outline:: Overall layout of the grammar file.
185* Symbols:: Terminal and nonterminal symbols.
186* Rules:: How to write grammar rules.
187* Recursion:: Writing recursive rules.
188* Semantics:: Semantic values and actions.
93dd49ab 189* Locations:: Locations and actions.
bfa74976
RS
190* Declarations:: All kinds of Bison declarations are described here.
191* Multiple Parsers:: Putting more than one Bison parser in one program.
192
193Outline of a Bison Grammar
194
f5f419de 195* Prologue:: Syntax and usage of the prologue.
2cbe6b7f 196* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
f5f419de
DJ
197* Bison Declarations:: Syntax and usage of the Bison declarations section.
198* Grammar Rules:: Syntax and usage of the grammar rules section.
199* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
200
201Defining Language Semantics
202
203* Value Type:: Specifying one data type for all semantic values.
204* Multiple Types:: Specifying several alternative data types.
205* Actions:: An action is the semantic definition of a grammar rule.
206* Action Types:: Specifying data types for actions to operate on.
207* Mid-Rule Actions:: Most actions go at the end of a rule.
208 This says when, why and how to use the exceptional
209 action in the middle of a rule.
d013372c 210* Named References:: Using named references in actions.
bfa74976 211
93dd49ab
PE
212Tracking Locations
213
214* Location Type:: Specifying a data type for locations.
215* Actions and Locations:: Using locations in actions.
216* Location Default Action:: Defining a general way to compute locations.
217
bfa74976
RS
218Bison Declarations
219
b50d2359 220* Require Decl:: Requiring a Bison version.
bfa74976
RS
221* Token Decl:: Declaring terminal symbols.
222* Precedence Decl:: Declaring terminals with precedence and associativity.
223* Union Decl:: Declaring the set of all semantic value types.
224* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 225* Initial Action Decl:: Code run before parsing starts.
72f889cc 226* Destructor Decl:: Declaring how symbols are freed.
d6328241 227* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
228* Start Decl:: Specifying the start symbol.
229* Pure Decl:: Requesting a reentrant parser.
9987d1b3 230* Push Decl:: Requesting a push parser.
bfa74976 231* Decl Summary:: Table of all Bison declarations.
35c1e5f0 232* %define Summary:: Defining variables to adjust Bison's behavior.
e0c07222 233* %code Summary:: Inserting code into the parser source.
bfa74976
RS
234
235Parser C-Language Interface
236
f5f419de
DJ
237* Parser Function:: How to call @code{yyparse} and what it returns.
238* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
239* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
240* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
241* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
242* Lexical:: You must supply a function @code{yylex}
243 which reads tokens.
244* Error Reporting:: You must supply a function @code{yyerror}.
245* Action Features:: Special features for use in actions.
246* Internationalization:: How to let the parser speak in the user's
247 native language.
bfa74976
RS
248
249The Lexical Analyzer Function @code{yylex}
250
251* Calling Convention:: How @code{yyparse} calls @code{yylex}.
f5f419de
DJ
252* Token Values:: How @code{yylex} must return the semantic value
253 of the token it has read.
254* Token Locations:: How @code{yylex} must return the text location
255 (line number, etc.) of the token, if the
256 actions want that.
257* Pure Calling:: How the calling convention differs in a pure parser
258 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
bfa74976 259
13863333 260The Bison Parser Algorithm
bfa74976 261
742e4900 262* Lookahead:: Parser looks one token ahead when deciding what to do.
bfa74976
RS
263* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
264* Precedence:: Operator precedence works by resolving conflicts.
265* Contextual Precedence:: When an operator's precedence depends on context.
266* Parser States:: The parser is a finite-state-machine with stack.
267* Reduce/Reduce:: When two rules are applicable in the same situation.
f5f419de 268* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 269* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 270* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
271
272Operator Precedence
273
274* Why Precedence:: An example showing why precedence is needed.
d78f0ac9
AD
275* Using Precedence:: How to specify precedence and associativity.
276* Precedence Only:: How to specify precedence only.
bfa74976
RS
277* Precedence Examples:: How these features are used in the previous example.
278* How Precedence:: How they work.
279
280Handling Context Dependencies
281
282* Semantic Tokens:: Token parsing can depend on the semantic context.
283* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
284* Tie-in Recovery:: Lexical tie-ins have implications for how
285 error recovery rules must be written.
286
93dd49ab 287Debugging Your Parser
ec3bc396
AD
288
289* Understanding:: Understanding the structure of your parser.
290* Tracing:: Tracing the execution of your parser.
291
bfa74976
RS
292Invoking Bison
293
13863333 294* Bison Options:: All the options described in detail,
c827f760 295 in alphabetical order by short options.
bfa74976 296* Option Cross Key:: Alphabetical list of long options.
93dd49ab 297* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
f2b5126e 298
8405b70c 299Parsers Written In Other Languages
12545799
AD
300
301* C++ Parsers:: The interface to generate C++ parser classes
8405b70c 302* Java Parsers:: The interface to generate Java parser classes
12545799
AD
303
304C++ Parsers
305
306* C++ Bison Interface:: Asking for C++ parser generation
307* C++ Semantic Values:: %union vs. C++
308* C++ Location Values:: The position and location classes
309* C++ Parser Interface:: Instantiating and running the parser
310* C++ Scanner Interface:: Exchanges between yylex and parse
8405b70c 311* A Complete C++ Example:: Demonstrating their use
12545799
AD
312
313A Complete C++ Example
314
315* Calc++ --- C++ Calculator:: The specifications
316* Calc++ Parsing Driver:: An active parsing context
317* Calc++ Parser:: A parser class
318* Calc++ Scanner:: A pure C++ Flex scanner
319* Calc++ Top Level:: Conducting the band
320
8405b70c
PB
321Java Parsers
322
f5f419de
DJ
323* Java Bison Interface:: Asking for Java parser generation
324* Java Semantic Values:: %type and %token vs. Java
325* Java Location Values:: The position and location classes
326* Java Parser Interface:: Instantiating and running the parser
327* Java Scanner Interface:: Specifying the scanner for the parser
328* Java Action Features:: Special features for use in actions
329* Java Differences:: Differences between C/C++ and Java Grammars
330* Java Declarations Summary:: List of Bison declarations used with Java
8405b70c 331
d1a1114f
AD
332Frequently Asked Questions
333
f5f419de
DJ
334* Memory Exhausted:: Breaking the Stack Limits
335* How Can I Reset the Parser:: @code{yyparse} Keeps some State
336* Strings are Destroyed:: @code{yylval} Loses Track of Strings
337* Implementing Gotos/Loops:: Control Flow in the Calculator
338* Multiple start-symbols:: Factoring closely related grammars
8a4281b9 339* Secure? Conform?:: Is Bison POSIX safe?
f5f419de
DJ
340* I can't build Bison:: Troubleshooting
341* Where can I find help?:: Troubleshouting
342* Bug Reports:: Troublereporting
343* More Languages:: Parsers in C++, Java, and so on
344* Beta Testing:: Experimenting development versions
345* Mailing Lists:: Meeting other Bison users
d1a1114f 346
f2b5126e
PB
347Copying This Manual
348
f5f419de 349* Copying This Manual:: License for copying this manual.
f2b5126e 350
342b8b6e 351@end detailmenu
bfa74976
RS
352@end menu
353
342b8b6e 354@node Introduction
bfa74976
RS
355@unnumbered Introduction
356@cindex introduction
357
6077da58 358@dfn{Bison} is a general-purpose parser generator that converts an
af28d414
JD
359annotated context-free grammar into a deterministic LR or generalized
360LR (GLR) parser employing LALR(1) parser tables. As an experimental
361feature, Bison can also generate IELR(1) or canonical LR(1) parser
362tables. Once you are proficient with Bison, you can use it to develop
363a wide range of language parsers, from those used in simple desk
364calculators to complex programming languages.
365
366Bison is upward compatible with Yacc: all properly-written Yacc
367grammars ought to work with Bison with no change. Anyone familiar
368with Yacc should be able to use Bison with little trouble. You need
369to be fluent in C or C++ programming in order to use Bison or to
370understand this manual. Java is also supported as an experimental
371feature.
372
373We begin with tutorial chapters that explain the basic concepts of
374using Bison and show three explained examples, each building on the
375last. If you don't know Bison or Yacc, start by reading these
376chapters. Reference chapters follow, which describe specific aspects
377of Bison in detail.
bfa74976 378
679e9935
JD
379Bison was written originally by Robert Corbett. Richard Stallman made
380it Yacc-compatible. Wilfred Hansen of Carnegie Mellon University
381added multi-character string literals and other features. Since then,
382Bison has grown more robust and evolved many other new features thanks
383to the hard work of a long list of volunteers. For details, see the
384@file{THANKS} and @file{ChangeLog} files included in the Bison
385distribution.
931c7513 386
df1af54c 387This edition corresponds to version @value{VERSION} of Bison.
bfa74976 388
342b8b6e 389@node Conditions
bfa74976
RS
390@unnumbered Conditions for Using Bison
391
193d7c70
PE
392The distribution terms for Bison-generated parsers permit using the
393parsers in nonfree programs. Before Bison version 2.2, these extra
8a4281b9 394permissions applied only when Bison was generating LALR(1)
193d7c70 395parsers in C@. And before Bison version 1.24, Bison-generated
262aa8dd 396parsers could be used only in programs that were free software.
a31239f1 397
8a4281b9 398The other GNU programming tools, such as the GNU C
c827f760 399compiler, have never
9ecbd125 400had such a requirement. They could always be used for nonfree
a31239f1
RS
401software. The reason Bison was different was not due to a special
402policy decision; it resulted from applying the usual General Public
403License to all of the Bison source code.
404
ff7571c0
JD
405The main output of the Bison utility---the Bison parser implementation
406file---contains a verbatim copy of a sizable piece of Bison, which is
407the code for the parser's implementation. (The actions from your
408grammar are inserted into this implementation at one point, but most
409of the rest of the implementation is not changed.) When we applied
410the GPL terms to the skeleton code for the parser's implementation,
a31239f1
RS
411the effect was to restrict the use of Bison output to free software.
412
413We didn't change the terms because of sympathy for people who want to
414make software proprietary. @strong{Software should be free.} But we
415concluded that limiting Bison's use to free software was doing little to
416encourage people to make other software free. So we decided to make the
417practical conditions for using Bison match the practical conditions for
8a4281b9 418using the other GNU tools.
bfa74976 419
193d7c70
PE
420This exception applies when Bison is generating code for a parser.
421You can tell whether the exception applies to a Bison output file by
422inspecting the file for text beginning with ``As a special
423exception@dots{}''. The text spells out the exact terms of the
424exception.
262aa8dd 425
f16b0819
PE
426@node Copying
427@unnumbered GNU GENERAL PUBLIC LICENSE
428@include gpl-3.0.texi
bfa74976 429
342b8b6e 430@node Concepts
bfa74976
RS
431@chapter The Concepts of Bison
432
433This chapter introduces many of the basic concepts without which the
434details of Bison will not make sense. If you do not already know how to
435use Bison or Yacc, we suggest you start by reading this chapter carefully.
436
437@menu
f5f419de
DJ
438* Language and Grammar:: Languages and context-free grammars,
439 as mathematical ideas.
440* Grammar in Bison:: How we represent grammars for Bison's sake.
441* Semantic Values:: Each token or syntactic grouping can have
442 a semantic value (the value of an integer,
443 the name of an identifier, etc.).
444* Semantic Actions:: Each rule can have an action containing C code.
445* GLR Parsers:: Writing parsers for general context-free languages.
446* Locations Overview:: Tracking Locations.
447* Bison Parser:: What are Bison's input and output,
448 how is the output used?
449* Stages:: Stages in writing and running Bison grammars.
450* Grammar Layout:: Overall structure of a Bison grammar file.
bfa74976
RS
451@end menu
452
342b8b6e 453@node Language and Grammar
bfa74976
RS
454@section Languages and Context-Free Grammars
455
bfa74976
RS
456@cindex context-free grammar
457@cindex grammar, context-free
458In order for Bison to parse a language, it must be described by a
459@dfn{context-free grammar}. This means that you specify one or more
460@dfn{syntactic groupings} and give rules for constructing them from their
461parts. For example, in the C language, one kind of grouping is called an
462`expression'. One rule for making an expression might be, ``An expression
463can be made of a minus sign and another expression''. Another would be,
464``An expression can be an integer''. As you can see, rules are often
465recursive, but there must be at least one rule which leads out of the
466recursion.
467
8a4281b9 468@cindex BNF
bfa74976
RS
469@cindex Backus-Naur form
470The most common formal system for presenting such rules for humans to read
8a4281b9 471is @dfn{Backus-Naur Form} or ``BNF'', which was developed in
c827f760 472order to specify the language Algol 60. Any grammar expressed in
8a4281b9
JD
473BNF is a context-free grammar. The input to Bison is
474essentially machine-readable BNF.
bfa74976 475
8a4281b9
JD
476@cindex LALR(1) grammars
477@cindex IELR(1) grammars
478@cindex LR(1) grammars
eb45ef3b
JD
479There are various important subclasses of context-free grammars.
480Although it can handle almost all context-free grammars, Bison is
8a4281b9 481optimized for what are called LR(1) grammars.
eb45ef3b
JD
482In brief, in these grammars, it must be possible to tell how to parse
483any portion of an input string with just a single token of lookahead.
484For historical reasons, Bison by default is limited by the additional
8a4281b9 485restrictions of LALR(1), which is hard to explain simply.
c827f760
PE
486@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
487more information on this.
f1b238df 488As an experimental feature, you can escape these additional restrictions by
8a4281b9 489requesting IELR(1) or canonical LR(1) parser tables.
35c1e5f0 490@xref{%define Summary,,lr.type}, to learn how.
bfa74976 491
8a4281b9
JD
492@cindex GLR parsing
493@cindex generalized LR (GLR) parsing
676385e2 494@cindex ambiguous grammars
9d9b8b70 495@cindex nondeterministic parsing
9501dc6e 496
8a4281b9 497Parsers for LR(1) grammars are @dfn{deterministic}, meaning
9501dc6e
AD
498roughly that the next grammar rule to apply at any point in the input is
499uniquely determined by the preceding input and a fixed, finite portion
742e4900 500(called a @dfn{lookahead}) of the remaining input. A context-free
9501dc6e 501grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
e4f85c39 502apply the grammar rules to get the same inputs. Even unambiguous
9d9b8b70 503grammars can be @dfn{nondeterministic}, meaning that no fixed
742e4900 504lookahead always suffices to determine the next grammar rule to apply.
9501dc6e 505With the proper declarations, Bison is also able to parse these more
8a4281b9
JD
506general context-free grammars, using a technique known as GLR
507parsing (for Generalized LR). Bison's GLR parsers
9501dc6e
AD
508are able to handle any context-free grammar for which the number of
509possible parses of any given string is finite.
676385e2 510
bfa74976
RS
511@cindex symbols (abstract)
512@cindex token
513@cindex syntactic grouping
514@cindex grouping, syntactic
9501dc6e
AD
515In the formal grammatical rules for a language, each kind of syntactic
516unit or grouping is named by a @dfn{symbol}. Those which are built by
517grouping smaller constructs according to grammatical rules are called
bfa74976
RS
518@dfn{nonterminal symbols}; those which can't be subdivided are called
519@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
520corresponding to a single terminal symbol a @dfn{token}, and a piece
e0c471a9 521corresponding to a single nonterminal symbol a @dfn{grouping}.
bfa74976
RS
522
523We can use the C language as an example of what symbols, terminal and
9501dc6e
AD
524nonterminal, mean. The tokens of C are identifiers, constants (numeric
525and string), and the various keywords, arithmetic operators and
526punctuation marks. So the terminal symbols of a grammar for C include
527`identifier', `number', `string', plus one symbol for each keyword,
528operator or punctuation mark: `if', `return', `const', `static', `int',
529`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
530(These tokens can be subdivided into characters, but that is a matter of
bfa74976
RS
531lexicography, not grammar.)
532
533Here is a simple C function subdivided into tokens:
534
9edcd895
AD
535@ifinfo
536@example
537int /* @r{keyword `int'} */
14d4662b 538square (int x) /* @r{identifier, open-paren, keyword `int',}
9edcd895
AD
539 @r{identifier, close-paren} */
540@{ /* @r{open-brace} */
aa08666d
AD
541 return x * x; /* @r{keyword `return', identifier, asterisk,}
542 @r{identifier, semicolon} */
9edcd895
AD
543@} /* @r{close-brace} */
544@end example
545@end ifinfo
546@ifnotinfo
bfa74976
RS
547@example
548int /* @r{keyword `int'} */
14d4662b 549square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
bfa74976 550@{ /* @r{open-brace} */
9edcd895 551 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
bfa74976
RS
552@} /* @r{close-brace} */
553@end example
9edcd895 554@end ifnotinfo
bfa74976
RS
555
556The syntactic groupings of C include the expression, the statement, the
557declaration, and the function definition. These are represented in the
558grammar of C by nonterminal symbols `expression', `statement',
559`declaration' and `function definition'. The full grammar uses dozens of
560additional language constructs, each with its own nonterminal symbol, in
561order to express the meanings of these four. The example above is a
562function definition; it contains one declaration, and one statement. In
563the statement, each @samp{x} is an expression and so is @samp{x * x}.
564
565Each nonterminal symbol must have grammatical rules showing how it is made
566out of simpler constructs. For example, one kind of C statement is the
567@code{return} statement; this would be described with a grammar rule which
568reads informally as follows:
569
570@quotation
571A `statement' can be made of a `return' keyword, an `expression' and a
572`semicolon'.
573@end quotation
574
575@noindent
576There would be many other rules for `statement', one for each kind of
577statement in C.
578
579@cindex start symbol
580One nonterminal symbol must be distinguished as the special one which
581defines a complete utterance in the language. It is called the @dfn{start
582symbol}. In a compiler, this means a complete input program. In the C
583language, the nonterminal symbol `sequence of definitions and declarations'
584plays this role.
585
586For example, @samp{1 + 2} is a valid C expression---a valid part of a C
587program---but it is not valid as an @emph{entire} C program. In the
588context-free grammar of C, this follows from the fact that `expression' is
589not the start symbol.
590
591The Bison parser reads a sequence of tokens as its input, and groups the
592tokens using the grammar rules. If the input is valid, the end result is
593that the entire token sequence reduces to a single grouping whose symbol is
594the grammar's start symbol. If we use a grammar for C, the entire input
595must be a `sequence of definitions and declarations'. If not, the parser
596reports a syntax error.
597
342b8b6e 598@node Grammar in Bison
bfa74976
RS
599@section From Formal Rules to Bison Input
600@cindex Bison grammar
601@cindex grammar, Bison
602@cindex formal grammar
603
604A formal grammar is a mathematical construct. To define the language
605for Bison, you must write a file expressing the grammar in Bison syntax:
606a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
607
608A nonterminal symbol in the formal grammar is represented in Bison input
c827f760 609as an identifier, like an identifier in C@. By convention, it should be
bfa74976
RS
610in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
611
612The Bison representation for a terminal symbol is also called a @dfn{token
613type}. Token types as well can be represented as C-like identifiers. By
614convention, these identifiers should be upper case to distinguish them from
615nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
616@code{RETURN}. A terminal symbol that stands for a particular keyword in
617the language should be named after that keyword converted to upper case.
618The terminal symbol @code{error} is reserved for error recovery.
931c7513 619@xref{Symbols}.
bfa74976
RS
620
621A terminal symbol can also be represented as a character literal, just like
622a C character constant. You should do this whenever a token is just a
623single character (parenthesis, plus-sign, etc.): use that same character in
624a literal as the terminal symbol for that token.
625
931c7513
RS
626A third way to represent a terminal symbol is with a C string constant
627containing several characters. @xref{Symbols}, for more information.
628
bfa74976
RS
629The grammar rules also have an expression in Bison syntax. For example,
630here is the Bison rule for a C @code{return} statement. The semicolon in
631quotes is a literal character token, representing part of the C syntax for
632the statement; the naked semicolon, and the colon, are Bison punctuation
633used in every rule.
634
635@example
636stmt: RETURN expr ';'
637 ;
638@end example
639
640@noindent
641@xref{Rules, ,Syntax of Grammar Rules}.
642
342b8b6e 643@node Semantic Values
bfa74976
RS
644@section Semantic Values
645@cindex semantic value
646@cindex value, semantic
647
648A formal grammar selects tokens only by their classifications: for example,
649if a rule mentions the terminal symbol `integer constant', it means that
650@emph{any} integer constant is grammatically valid in that position. The
651precise value of the constant is irrelevant to how to parse the input: if
652@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
e0c471a9 653grammatical.
bfa74976
RS
654
655But the precise value is very important for what the input means once it is
656parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6573989 as constants in the program! Therefore, each token in a Bison grammar
c827f760
PE
658has both a token type and a @dfn{semantic value}. @xref{Semantics,
659,Defining Language Semantics},
bfa74976
RS
660for details.
661
662The token type is a terminal symbol defined in the grammar, such as
663@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
664you need to know to decide where the token may validly appear and how to
665group it with other tokens. The grammar rules know nothing about tokens
e0c471a9 666except their types.
bfa74976
RS
667
668The semantic value has all the rest of the information about the
669meaning of the token, such as the value of an integer, or the name of an
670identifier. (A token such as @code{','} which is just punctuation doesn't
671need to have any semantic value.)
672
673For example, an input token might be classified as token type
674@code{INTEGER} and have the semantic value 4. Another input token might
675have the same token type @code{INTEGER} but value 3989. When a grammar
676rule says that @code{INTEGER} is allowed, either of these tokens is
677acceptable because each is an @code{INTEGER}. When the parser accepts the
678token, it keeps track of the token's semantic value.
679
680Each grouping can also have a semantic value as well as its nonterminal
681symbol. For example, in a calculator, an expression typically has a
682semantic value that is a number. In a compiler for a programming
683language, an expression typically has a semantic value that is a tree
684structure describing the meaning of the expression.
685
342b8b6e 686@node Semantic Actions
bfa74976
RS
687@section Semantic Actions
688@cindex semantic actions
689@cindex actions, semantic
690
691In order to be useful, a program must do more than parse input; it must
692also produce some output based on the input. In a Bison grammar, a grammar
693rule can have an @dfn{action} made up of C statements. Each time the
694parser recognizes a match for that rule, the action is executed.
695@xref{Actions}.
13863333 696
bfa74976
RS
697Most of the time, the purpose of an action is to compute the semantic value
698of the whole construct from the semantic values of its parts. For example,
699suppose we have a rule which says an expression can be the sum of two
700expressions. When the parser recognizes such a sum, each of the
701subexpressions has a semantic value which describes how it was built up.
702The action for this rule should create a similar sort of value for the
703newly recognized larger expression.
704
705For example, here is a rule that says an expression can be the sum of
706two subexpressions:
707
708@example
709expr: expr '+' expr @{ $$ = $1 + $3; @}
710 ;
711@end example
712
713@noindent
714The action says how to produce the semantic value of the sum expression
715from the values of the two subexpressions.
716
676385e2 717@node GLR Parsers
8a4281b9
JD
718@section Writing GLR Parsers
719@cindex GLR parsing
720@cindex generalized LR (GLR) parsing
676385e2
PH
721@findex %glr-parser
722@cindex conflicts
723@cindex shift/reduce conflicts
fa7e68c3 724@cindex reduce/reduce conflicts
676385e2 725
eb45ef3b 726In some grammars, Bison's deterministic
8a4281b9 727LR(1) parsing algorithm cannot decide whether to apply a
9501dc6e
AD
728certain grammar rule at a given point. That is, it may not be able to
729decide (on the basis of the input read so far) which of two possible
730reductions (applications of a grammar rule) applies, or whether to apply
731a reduction or read more of the input and apply a reduction later in the
732input. These are known respectively as @dfn{reduce/reduce} conflicts
733(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
734(@pxref{Shift/Reduce}).
735
8a4281b9 736To use a grammar that is not easily modified to be LR(1), a
9501dc6e 737more general parsing algorithm is sometimes necessary. If you include
676385e2 738@code{%glr-parser} among the Bison declarations in your file
8a4281b9
JD
739(@pxref{Grammar Outline}), the result is a Generalized LR
740(GLR) parser. These parsers handle Bison grammars that
9501dc6e 741contain no unresolved conflicts (i.e., after applying precedence
eb45ef3b 742declarations) identically to deterministic parsers. However, when
9501dc6e 743faced with unresolved shift/reduce and reduce/reduce conflicts,
8a4281b9 744GLR parsers use the simple expedient of doing both,
9501dc6e
AD
745effectively cloning the parser to follow both possibilities. Each of
746the resulting parsers can again split, so that at any given time, there
747can be any number of possible parses being explored. The parsers
676385e2
PH
748proceed in lockstep; that is, all of them consume (shift) a given input
749symbol before any of them proceed to the next. Each of the cloned
750parsers eventually meets one of two possible fates: either it runs into
751a parsing error, in which case it simply vanishes, or it merges with
752another parser, because the two of them have reduced the input to an
753identical set of symbols.
754
755During the time that there are multiple parsers, semantic actions are
756recorded, but not performed. When a parser disappears, its recorded
757semantic actions disappear as well, and are never performed. When a
758reduction makes two parsers identical, causing them to merge, Bison
759records both sets of semantic actions. Whenever the last two parsers
760merge, reverting to the single-parser case, Bison resolves all the
761outstanding actions either by precedences given to the grammar rules
762involved, or by performing both actions, and then calling a designated
763user-defined function on the resulting values to produce an arbitrary
764merged result.
765
fa7e68c3 766@menu
8a4281b9
JD
767* Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
768* Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
20be2f92 769* GLR Semantic Actions:: Considerations for semantic values and deferred actions.
ca2a6d15 770* Semantic Predicates:: Controlling a parse with arbitrary computations.
8a4281b9 771* Compiler Requirements:: GLR parsers require a modern C compiler.
fa7e68c3
PE
772@end menu
773
774@node Simple GLR Parsers
8a4281b9
JD
775@subsection Using GLR on Unambiguous Grammars
776@cindex GLR parsing, unambiguous grammars
777@cindex generalized LR (GLR) parsing, unambiguous grammars
fa7e68c3
PE
778@findex %glr-parser
779@findex %expect-rr
780@cindex conflicts
781@cindex reduce/reduce conflicts
782@cindex shift/reduce conflicts
783
8a4281b9
JD
784In the simplest cases, you can use the GLR algorithm
785to parse grammars that are unambiguous but fail to be LR(1).
eb45ef3b 786Such grammars typically require more than one symbol of lookahead.
fa7e68c3
PE
787
788Consider a problem that
789arises in the declaration of enumerated and subrange types in the
790programming language Pascal. Here are some examples:
791
792@example
793type subrange = lo .. hi;
794type enum = (a, b, c);
795@end example
796
797@noindent
798The original language standard allows only numeric
799literals and constant identifiers for the subrange bounds (@samp{lo}
8a4281b9 800and @samp{hi}), but Extended Pascal (ISO/IEC
fa7e68c3
PE
80110206) and many other
802Pascal implementations allow arbitrary expressions there. This gives
803rise to the following situation, containing a superfluous pair of
804parentheses:
805
806@example
807type subrange = (a) .. b;
808@end example
809
810@noindent
811Compare this to the following declaration of an enumerated
812type with only one value:
813
814@example
815type enum = (a);
816@end example
817
818@noindent
819(These declarations are contrived, but they are syntactically
820valid, and more-complicated cases can come up in practical programs.)
821
822These two declarations look identical until the @samp{..} token.
8a4281b9 823With normal LR(1) one-token lookahead it is not
fa7e68c3
PE
824possible to decide between the two forms when the identifier
825@samp{a} is parsed. It is, however, desirable
826for a parser to decide this, since in the latter case
827@samp{a} must become a new identifier to represent the enumeration
828value, while in the former case @samp{a} must be evaluated with its
829current meaning, which may be a constant or even a function call.
830
831You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
832to be resolved later, but this typically requires substantial
833contortions in both semantic actions and large parts of the
834grammar, where the parentheses are nested in the recursive rules for
835expressions.
836
837You might think of using the lexer to distinguish between the two
838forms by returning different tokens for currently defined and
839undefined identifiers. But if these declarations occur in a local
840scope, and @samp{a} is defined in an outer scope, then both forms
841are possible---either locally redefining @samp{a}, or using the
842value of @samp{a} from the outer scope. So this approach cannot
843work.
844
e757bb10 845A simple solution to this problem is to declare the parser to
8a4281b9
JD
846use the GLR algorithm.
847When the GLR parser reaches the critical state, it
fa7e68c3
PE
848merely splits into two branches and pursues both syntax rules
849simultaneously. Sooner or later, one of them runs into a parsing
850error. If there is a @samp{..} token before the next
851@samp{;}, the rule for enumerated types fails since it cannot
852accept @samp{..} anywhere; otherwise, the subrange type rule
853fails since it requires a @samp{..} token. So one of the branches
854fails silently, and the other one continues normally, performing
855all the intermediate actions that were postponed during the split.
856
857If the input is syntactically incorrect, both branches fail and the parser
858reports a syntax error as usual.
859
860The effect of all this is that the parser seems to ``guess'' the
861correct branch to take, or in other words, it seems to use more
8a4281b9
JD
862lookahead than the underlying LR(1) algorithm actually allows
863for. In this example, LR(2) would suffice, but also some cases
864that are not LR(@math{k}) for any @math{k} can be handled this way.
fa7e68c3 865
8a4281b9 866In general, a GLR parser can take quadratic or cubic worst-case time,
fa7e68c3
PE
867and the current Bison parser even takes exponential time and space
868for some grammars. In practice, this rarely happens, and for many
869grammars it is possible to prove that it cannot happen.
870The present example contains only one conflict between two
871rules, and the type-declaration context containing the conflict
872cannot be nested. So the number of
873branches that can exist at any time is limited by the constant 2,
874and the parsing time is still linear.
875
876Here is a Bison grammar corresponding to the example above. It
877parses a vastly simplified form of Pascal type declarations.
878
879@example
880%token TYPE DOTDOT ID
881
882@group
883%left '+' '-'
884%left '*' '/'
885@end group
886
887%%
888
889@group
890type_decl : TYPE ID '=' type ';'
891 ;
892@end group
893
894@group
895type : '(' id_list ')'
896 | expr DOTDOT expr
897 ;
898@end group
899
900@group
901id_list : ID
902 | id_list ',' ID
903 ;
904@end group
905
906@group
907expr : '(' expr ')'
908 | expr '+' expr
909 | expr '-' expr
910 | expr '*' expr
911 | expr '/' expr
912 | ID
913 ;
914@end group
915@end example
916
8a4281b9 917When used as a normal LR(1) grammar, Bison correctly complains
fa7e68c3
PE
918about one reduce/reduce conflict. In the conflicting situation the
919parser chooses one of the alternatives, arbitrarily the one
920declared first. Therefore the following correct input is not
921recognized:
922
923@example
924type t = (a) .. b;
925@end example
926
8a4281b9 927The parser can be turned into a GLR parser, while also telling Bison
ff7571c0
JD
928to be silent about the one known reduce/reduce conflict, by adding
929these two declarations to the Bison grammar file (before the first
fa7e68c3
PE
930@samp{%%}):
931
932@example
933%glr-parser
934%expect-rr 1
935@end example
936
937@noindent
938No change in the grammar itself is required. Now the
939parser recognizes all valid declarations, according to the
940limited syntax above, transparently. In fact, the user does not even
941notice when the parser splits.
942
8a4281b9 943So here we have a case where we can use the benefits of GLR,
f8e1c9e5
AD
944almost without disadvantages. Even in simple cases like this, however,
945there are at least two potential problems to beware. First, always
8a4281b9
JD
946analyze the conflicts reported by Bison to make sure that GLR
947splitting is only done where it is intended. A GLR parser
f8e1c9e5 948splitting inadvertently may cause problems less obvious than an
8a4281b9 949LR parser statically choosing the wrong alternative in a
f8e1c9e5
AD
950conflict. Second, consider interactions with the lexer (@pxref{Semantic
951Tokens}) with great care. Since a split parser consumes tokens without
952performing any actions during the split, the lexer cannot obtain
953information via parser actions. Some cases of lexer interactions can be
8a4281b9 954eliminated by using GLR to shift the complications from the
f8e1c9e5
AD
955lexer to the parser. You must check the remaining cases for
956correctness.
957
958In our example, it would be safe for the lexer to return tokens based on
959their current meanings in some symbol table, because no new symbols are
960defined in the middle of a type declaration. Though it is possible for
961a parser to define the enumeration constants as they are parsed, before
962the type declaration is completed, it actually makes no difference since
963they cannot be used within the same enumerated type declaration.
fa7e68c3
PE
964
965@node Merging GLR Parses
8a4281b9
JD
966@subsection Using GLR to Resolve Ambiguities
967@cindex GLR parsing, ambiguous grammars
968@cindex generalized LR (GLR) parsing, ambiguous grammars
fa7e68c3
PE
969@findex %dprec
970@findex %merge
971@cindex conflicts
972@cindex reduce/reduce conflicts
973
2a8d363a 974Let's consider an example, vastly simplified from a C++ grammar.
676385e2
PH
975
976@example
977%@{
38a92d50
PE
978 #include <stdio.h>
979 #define YYSTYPE char const *
980 int yylex (void);
981 void yyerror (char const *);
676385e2
PH
982%@}
983
984%token TYPENAME ID
985
986%right '='
987%left '+'
988
989%glr-parser
990
991%%
992
fae437e8 993prog :
676385e2
PH
994 | prog stmt @{ printf ("\n"); @}
995 ;
996
997stmt : expr ';' %dprec 1
998 | decl %dprec 2
999 ;
1000
2a8d363a 1001expr : ID @{ printf ("%s ", $$); @}
fae437e8 1002 | TYPENAME '(' expr ')'
2a8d363a
AD
1003 @{ printf ("%s <cast> ", $1); @}
1004 | expr '+' expr @{ printf ("+ "); @}
1005 | expr '=' expr @{ printf ("= "); @}
676385e2
PH
1006 ;
1007
fae437e8 1008decl : TYPENAME declarator ';'
2a8d363a 1009 @{ printf ("%s <declare> ", $1); @}
676385e2 1010 | TYPENAME declarator '=' expr ';'
2a8d363a 1011 @{ printf ("%s <init-declare> ", $1); @}
676385e2
PH
1012 ;
1013
2a8d363a 1014declarator : ID @{ printf ("\"%s\" ", $1); @}
676385e2
PH
1015 | '(' declarator ')'
1016 ;
1017@end example
1018
1019@noindent
1020This models a problematic part of the C++ grammar---the ambiguity between
1021certain declarations and statements. For example,
1022
1023@example
1024T (x) = y+z;
1025@end example
1026
1027@noindent
1028parses as either an @code{expr} or a @code{stmt}
c827f760
PE
1029(assuming that @samp{T} is recognized as a @code{TYPENAME} and
1030@samp{x} as an @code{ID}).
676385e2 1031Bison detects this as a reduce/reduce conflict between the rules
fae437e8 1032@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
e757bb10 1033time it encounters @code{x} in the example above. Since this is a
8a4281b9 1034GLR parser, it therefore splits the problem into two parses, one for
fa7e68c3
PE
1035each choice of resolving the reduce/reduce conflict.
1036Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1037however, neither of these parses ``dies,'' because the grammar as it stands is
e757bb10
AD
1038ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1039the other reduces @code{stmt : decl}, after which both parsers are in an
1040identical state: they've seen @samp{prog stmt} and have the same unprocessed
1041input remaining. We say that these parses have @dfn{merged.}
fa7e68c3 1042
8a4281b9 1043At this point, the GLR parser requires a specification in the
fa7e68c3
PE
1044grammar of how to choose between the competing parses.
1045In the example above, the two @code{%dprec}
e757bb10 1046declarations specify that Bison is to give precedence
fa7e68c3 1047to the parse that interprets the example as a
676385e2
PH
1048@code{decl}, which implies that @code{x} is a declarator.
1049The parser therefore prints
1050
1051@example
fae437e8 1052"x" y z + T <init-declare>
676385e2
PH
1053@end example
1054
fa7e68c3
PE
1055The @code{%dprec} declarations only come into play when more than one
1056parse survives. Consider a different input string for this parser:
676385e2
PH
1057
1058@example
1059T (x) + y;
1060@end example
1061
1062@noindent
8a4281b9 1063This is another example of using GLR to parse an unambiguous
fa7e68c3 1064construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
676385e2
PH
1065Here, there is no ambiguity (this cannot be parsed as a declaration).
1066However, at the time the Bison parser encounters @code{x}, it does not
1067have enough information to resolve the reduce/reduce conflict (again,
1068between @code{x} as an @code{expr} or a @code{declarator}). In this
fa7e68c3 1069case, no precedence declaration is used. Again, the parser splits
676385e2
PH
1070into two, one assuming that @code{x} is an @code{expr}, and the other
1071assuming @code{x} is a @code{declarator}. The second of these parsers
1072then vanishes when it sees @code{+}, and the parser prints
1073
1074@example
fae437e8 1075x T <cast> y +
676385e2
PH
1076@end example
1077
1078Suppose that instead of resolving the ambiguity, you wanted to see all
fa7e68c3 1079the possibilities. For this purpose, you must merge the semantic
676385e2
PH
1080actions of the two possible parsers, rather than choosing one over the
1081other. To do so, you could change the declaration of @code{stmt} as
1082follows:
1083
1084@example
1085stmt : expr ';' %merge <stmtMerge>
1086 | decl %merge <stmtMerge>
1087 ;
1088@end example
1089
1090@noindent
676385e2
PH
1091and define the @code{stmtMerge} function as:
1092
1093@example
38a92d50
PE
1094static YYSTYPE
1095stmtMerge (YYSTYPE x0, YYSTYPE x1)
676385e2
PH
1096@{
1097 printf ("<OR> ");
1098 return "";
1099@}
1100@end example
1101
1102@noindent
1103with an accompanying forward declaration
1104in the C declarations at the beginning of the file:
1105
1106@example
1107%@{
38a92d50 1108 #define YYSTYPE char const *
676385e2
PH
1109 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1110%@}
1111@end example
1112
1113@noindent
fa7e68c3
PE
1114With these declarations, the resulting parser parses the first example
1115as both an @code{expr} and a @code{decl}, and prints
676385e2
PH
1116
1117@example
fae437e8 1118"x" y z + T <init-declare> x T <cast> y z + = <OR>
676385e2
PH
1119@end example
1120
fa7e68c3 1121Bison requires that all of the
e757bb10 1122productions that participate in any particular merge have identical
fa7e68c3
PE
1123@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1124and the parser will report an error during any parse that results in
1125the offending merge.
9501dc6e 1126
32c29292
JD
1127@node GLR Semantic Actions
1128@subsection GLR Semantic Actions
1129
8a4281b9 1130The nature of GLR parsing and the structure of the generated
20be2f92
PH
1131parsers give rise to certain restrictions on semantic values and actions.
1132
1133@subsubsection Deferred semantic actions
32c29292
JD
1134@cindex deferred semantic actions
1135By definition, a deferred semantic action is not performed at the same time as
1136the associated reduction.
1137This raises caveats for several Bison features you might use in a semantic
8a4281b9 1138action in a GLR parser.
32c29292
JD
1139
1140@vindex yychar
8a4281b9 1141@cindex GLR parsers and @code{yychar}
32c29292 1142@vindex yylval
8a4281b9 1143@cindex GLR parsers and @code{yylval}
32c29292 1144@vindex yylloc
8a4281b9 1145@cindex GLR parsers and @code{yylloc}
32c29292 1146In any semantic action, you can examine @code{yychar} to determine the type of
742e4900 1147the lookahead token present at the time of the associated reduction.
32c29292
JD
1148After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1149you can then examine @code{yylval} and @code{yylloc} to determine the
742e4900 1150lookahead token's semantic value and location, if any.
32c29292
JD
1151In a nondeferred semantic action, you can also modify any of these variables to
1152influence syntax analysis.
742e4900 1153@xref{Lookahead, ,Lookahead Tokens}.
32c29292
JD
1154
1155@findex yyclearin
8a4281b9 1156@cindex GLR parsers and @code{yyclearin}
32c29292
JD
1157In a deferred semantic action, it's too late to influence syntax analysis.
1158In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1159shallow copies of the values they had at the time of the associated reduction.
1160For this reason alone, modifying them is dangerous.
1161Moreover, the result of modifying them is undefined and subject to change with
1162future versions of Bison.
1163For example, if a semantic action might be deferred, you should never write it
1164to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1165memory referenced by @code{yylval}.
1166
20be2f92 1167@subsubsection YYERROR
32c29292 1168@findex YYERROR
8a4281b9 1169@cindex GLR parsers and @code{YYERROR}
32c29292 1170Another Bison feature requiring special consideration is @code{YYERROR}
8710fc41 1171(@pxref{Action Features}), which you can invoke in a semantic action to
32c29292 1172initiate error recovery.
8a4281b9 1173During deterministic GLR operation, the effect of @code{YYERROR} is
eb45ef3b 1174the same as its effect in a deterministic parser.
20be2f92
PH
1175The effect in a deferred action is similar, but the precise point of the
1176error is undefined; instead, the parser reverts to deterministic operation,
1177selecting an unspecified stack on which to continue with a syntax error.
1178In a semantic predicate (see @ref{Semantic Predicates}) during nondeterministic
1179parsing, @code{YYERROR} silently prunes
1180the parse that invoked the test.
1181
1182@subsubsection Restrictions on semantic values and locations
8a4281b9 1183GLR parsers require that you use POD (Plain Old Data) types for
20be2f92
PH
1184semantic values and location types when using the generated parsers as
1185C++ code.
8710fc41 1186
ca2a6d15
PH
1187@node Semantic Predicates
1188@subsection Controlling a Parse with Arbitrary Predicates
1189@findex %?
8a4281b9 1190@cindex Semantic predicates in GLR parsers
ca2a6d15
PH
1191
1192In addition to the @code{%dprec} and @code{%merge} directives,
8a4281b9 1193GLR parsers
ca2a6d15
PH
1194allow you to reject parses on the basis of arbitrary computations executed
1195in user code, without having Bison treat this rejection as an error
1196if there are alternative parses. (This feature is experimental and may
1197evolve. We welcome user feedback.) For example,
1198
1199@smallexample
1200widget :
1201 %?@{ new_syntax @} "widget" id new_args @{ $$ = f($3, $4); @}
1202 | %?@{ !new_syntax @} "widget" id old_args @{ $$ = f($3, $4); @}
1203 ;
1204@end smallexample
1205
1206@noindent
1207is one way to allow the same parser to handle two different syntaxes for
1208widgets. The clause preceded by @code{%?} is treated like an ordinary
1209action, except that its text is treated as an expression and is always
1210evaluated immediately (even when in nondeterministic mode). If the
1211expression yields 0 (false), the clause is treated as a syntax error,
1212which, in a nondeterministic parser, causes the stack in which it is reduced
1213to die. In a deterministic parser, it acts like YYERROR.
1214
1215As the example shows, predicates otherwise look like semantic actions, and
1216therefore you must be take them into account when determining the numbers
1217to use for denoting the semantic values of right-hand side symbols.
1218Predicate actions, however, have no defined value, and may not be given
1219labels.
1220
1221There is a subtle difference between semantic predicates and ordinary
1222actions in nondeterministic mode, since the latter are deferred.
1223For example, we could try to rewrite the previous example as
1224
1225@smallexample
1226widget :
1227 @{ if (!new_syntax) YYERROR; @} "widget" id new_args @{ $$ = f($3, $4); @}
1228 | @{ if (new_syntax) YYERROR; @} "widget" id old_args @{ $$ = f($3, $4); @}
1229 ;
1230@end smallexample
1231
1232@noindent
1233(reversing the sense of the predicate tests to cause an error when they are
1234false). However, this
1235does @emph{not} have the same effect if @code{new_args} and @code{old_args}
1236have overlapping syntax.
1237Since the mid-rule actions testing @code{new_syntax} are deferred,
8a4281b9 1238a GLR parser first encounters the unresolved ambiguous reduction
ca2a6d15
PH
1239for cases where @code{new_args} and @code{old_args} recognize the same string
1240@emph{before} performing the tests of @code{new_syntax}. It therefore
1241reports an error.
1242
1243Finally, be careful in writing predicates: deferred actions have not been
1244evaluated, so that using them in a predicate will have undefined effects.
1245
fa7e68c3 1246@node Compiler Requirements
8a4281b9 1247@subsection Considerations when Compiling GLR Parsers
fa7e68c3 1248@cindex @code{inline}
8a4281b9 1249@cindex GLR parsers and @code{inline}
fa7e68c3 1250
8a4281b9 1251The GLR parsers require a compiler for ISO C89 or
38a92d50
PE
1252later. In addition, they use the @code{inline} keyword, which is not
1253C89, but is C99 and is a common extension in pre-C99 compilers. It is
1254up to the user of these parsers to handle
9501dc6e
AD
1255portability issues. For instance, if using Autoconf and the Autoconf
1256macro @code{AC_C_INLINE}, a mere
1257
1258@example
1259%@{
38a92d50 1260 #include <config.h>
9501dc6e
AD
1261%@}
1262@end example
1263
1264@noindent
1265will suffice. Otherwise, we suggest
1266
1267@example
1268%@{
38a92d50
PE
1269 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1270 #define inline
1271 #endif
9501dc6e
AD
1272%@}
1273@end example
676385e2 1274
342b8b6e 1275@node Locations Overview
847bf1f5
AD
1276@section Locations
1277@cindex location
95923bd6
AD
1278@cindex textual location
1279@cindex location, textual
847bf1f5
AD
1280
1281Many applications, like interpreters or compilers, have to produce verbose
72d2299c 1282and useful error messages. To achieve this, one must be able to keep track of
95923bd6 1283the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
847bf1f5
AD
1284Bison provides a mechanism for handling these locations.
1285
72d2299c 1286Each token has a semantic value. In a similar fashion, each token has an
847bf1f5 1287associated location, but the type of locations is the same for all tokens and
72d2299c 1288groupings. Moreover, the output parser is equipped with a default data
847bf1f5
AD
1289structure for storing locations (@pxref{Locations}, for more details).
1290
1291Like semantic values, locations can be reached in actions using a dedicated
72d2299c 1292set of constructs. In the example above, the location of the whole grouping
847bf1f5
AD
1293is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1294@code{@@3}.
1295
1296When a rule is matched, a default action is used to compute the semantic value
72d2299c
PE
1297of its left hand side (@pxref{Actions}). In the same way, another default
1298action is used for locations. However, the action for locations is general
847bf1f5 1299enough for most cases, meaning there is usually no need to describe for each
72d2299c 1300rule how @code{@@$} should be formed. When building a new location for a given
847bf1f5
AD
1301grouping, the default behavior of the output parser is to take the beginning
1302of the first symbol, and the end of the last symbol.
1303
342b8b6e 1304@node Bison Parser
ff7571c0 1305@section Bison Output: the Parser Implementation File
bfa74976
RS
1306@cindex Bison parser
1307@cindex Bison utility
1308@cindex lexical analyzer, purpose
1309@cindex parser
1310
ff7571c0
JD
1311When you run Bison, you give it a Bison grammar file as input. The
1312most important output is a C source file that implements a parser for
1313the language described by the grammar. This parser is called a
1314@dfn{Bison parser}, and this file is called a @dfn{Bison parser
1315implementation file}. Keep in mind that the Bison utility and the
1316Bison parser are two distinct programs: the Bison utility is a program
1317whose output is the Bison parser implementation file that becomes part
1318of your program.
bfa74976
RS
1319
1320The job of the Bison parser is to group tokens into groupings according to
1321the grammar rules---for example, to build identifiers and operators into
1322expressions. As it does this, it runs the actions for the grammar rules it
1323uses.
1324
704a47c4
AD
1325The tokens come from a function called the @dfn{lexical analyzer} that
1326you must supply in some fashion (such as by writing it in C). The Bison
1327parser calls the lexical analyzer each time it wants a new token. It
1328doesn't know what is ``inside'' the tokens (though their semantic values
1329may reflect this). Typically the lexical analyzer makes the tokens by
1330parsing characters of text, but Bison does not depend on this.
1331@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
bfa74976 1332
ff7571c0
JD
1333The Bison parser implementation file is C code which defines a
1334function named @code{yyparse} which implements that grammar. This
1335function does not make a complete C program: you must supply some
1336additional functions. One is the lexical analyzer. Another is an
1337error-reporting function which the parser calls to report an error.
1338In addition, a complete C program must start with a function called
1339@code{main}; you have to provide this, and arrange for it to call
1340@code{yyparse} or the parser will never run. @xref{Interface, ,Parser
1341C-Language Interface}.
bfa74976 1342
f7ab6a50 1343Aside from the token type names and the symbols in the actions you
ff7571c0
JD
1344write, all symbols defined in the Bison parser implementation file
1345itself begin with @samp{yy} or @samp{YY}. This includes interface
1346functions such as the lexical analyzer function @code{yylex}, the
1347error reporting function @code{yyerror} and the parser function
1348@code{yyparse} itself. This also includes numerous identifiers used
1349for internal purposes. Therefore, you should avoid using C
1350identifiers starting with @samp{yy} or @samp{YY} in the Bison grammar
1351file except for the ones defined in this manual. Also, you should
1352avoid using the C identifiers @samp{malloc} and @samp{free} for
1353anything other than their usual meanings.
1354
1355In some cases the Bison parser implementation file includes system
1356headers, and in those cases your code should respect the identifiers
1357reserved by those headers. On some non-GNU hosts, @code{<alloca.h>},
1358@code{<malloc.h>}, @code{<stddef.h>}, and @code{<stdlib.h>} are
1359included as needed to declare memory allocators and related types.
1360@code{<libintl.h>} is included if message translation is in use
1361(@pxref{Internationalization}). Other system headers may be included
1362if you define @code{YYDEBUG} to a nonzero value (@pxref{Tracing,
1363,Tracing Your Parser}).
7093d0f5 1364
342b8b6e 1365@node Stages
bfa74976
RS
1366@section Stages in Using Bison
1367@cindex stages in using Bison
1368@cindex using Bison
1369
1370The actual language-design process using Bison, from grammar specification
1371to a working compiler or interpreter, has these parts:
1372
1373@enumerate
1374@item
1375Formally specify the grammar in a form recognized by Bison
704a47c4
AD
1376(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1377in the language, describe the action that is to be taken when an
1378instance of that rule is recognized. The action is described by a
1379sequence of C statements.
bfa74976
RS
1380
1381@item
704a47c4
AD
1382Write a lexical analyzer to process input and pass tokens to the parser.
1383The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1384Lexical Analyzer Function @code{yylex}}). It could also be produced
1385using Lex, but the use of Lex is not discussed in this manual.
bfa74976
RS
1386
1387@item
1388Write a controlling function that calls the Bison-produced parser.
1389
1390@item
1391Write error-reporting routines.
1392@end enumerate
1393
1394To turn this source code as written into a runnable program, you
1395must follow these steps:
1396
1397@enumerate
1398@item
1399Run Bison on the grammar to produce the parser.
1400
1401@item
1402Compile the code output by Bison, as well as any other source files.
1403
1404@item
1405Link the object files to produce the finished product.
1406@end enumerate
1407
342b8b6e 1408@node Grammar Layout
bfa74976
RS
1409@section The Overall Layout of a Bison Grammar
1410@cindex grammar file
1411@cindex file format
1412@cindex format of grammar file
1413@cindex layout of Bison grammar
1414
1415The input file for the Bison utility is a @dfn{Bison grammar file}. The
1416general form of a Bison grammar file is as follows:
1417
1418@example
1419%@{
08e49d20 1420@var{Prologue}
bfa74976
RS
1421%@}
1422
1423@var{Bison declarations}
1424
1425%%
1426@var{Grammar rules}
1427%%
08e49d20 1428@var{Epilogue}
bfa74976
RS
1429@end example
1430
1431@noindent
1432The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1433in every Bison grammar file to separate the sections.
1434
72d2299c 1435The prologue may define types and variables used in the actions. You can
342b8b6e 1436also use preprocessor commands to define macros used there, and use
bfa74976 1437@code{#include} to include header files that do any of these things.
38a92d50
PE
1438You need to declare the lexical analyzer @code{yylex} and the error
1439printer @code{yyerror} here, along with any other global identifiers
1440used by the actions in the grammar rules.
bfa74976
RS
1441
1442The Bison declarations declare the names of the terminal and nonterminal
1443symbols, and may also describe operator precedence and the data types of
1444semantic values of various symbols.
1445
1446The grammar rules define how to construct each nonterminal symbol from its
1447parts.
1448
38a92d50
PE
1449The epilogue can contain any code you want to use. Often the
1450definitions of functions declared in the prologue go here. In a
1451simple program, all the rest of the program can go here.
bfa74976 1452
342b8b6e 1453@node Examples
bfa74976
RS
1454@chapter Examples
1455@cindex simple examples
1456@cindex examples, simple
1457
1458Now we show and explain three sample programs written using Bison: a
1459reverse polish notation calculator, an algebraic (infix) notation
1460calculator, and a multi-function calculator. All three have been tested
1461under BSD Unix 4.3; each produces a usable, though limited, interactive
1462desk-top calculator.
1463
1464These examples are simple, but Bison grammars for real programming
aa08666d
AD
1465languages are written the same way. You can copy these examples into a
1466source file to try them.
bfa74976
RS
1467
1468@menu
f5f419de
DJ
1469* RPN Calc:: Reverse polish notation calculator;
1470 a first example with no operator precedence.
1471* Infix Calc:: Infix (algebraic) notation calculator.
1472 Operator precedence is introduced.
bfa74976 1473* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 1474* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
f5f419de
DJ
1475* Multi-function Calc:: Calculator with memory and trig functions.
1476 It uses multiple data-types for semantic values.
1477* Exercises:: Ideas for improving the multi-function calculator.
bfa74976
RS
1478@end menu
1479
342b8b6e 1480@node RPN Calc
bfa74976
RS
1481@section Reverse Polish Notation Calculator
1482@cindex reverse polish notation
1483@cindex polish notation calculator
1484@cindex @code{rpcalc}
1485@cindex calculator, simple
1486
1487The first example is that of a simple double-precision @dfn{reverse polish
1488notation} calculator (a calculator using postfix operators). This example
1489provides a good starting point, since operator precedence is not an issue.
1490The second example will illustrate how operator precedence is handled.
1491
1492The source code for this calculator is named @file{rpcalc.y}. The
ff7571c0 1493@samp{.y} extension is a convention used for Bison grammar files.
bfa74976
RS
1494
1495@menu
f5f419de
DJ
1496* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1497* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1498* Rpcalc Lexer:: The lexical analyzer.
1499* Rpcalc Main:: The controlling function.
1500* Rpcalc Error:: The error reporting function.
1501* Rpcalc Generate:: Running Bison on the grammar file.
1502* Rpcalc Compile:: Run the C compiler on the output code.
bfa74976
RS
1503@end menu
1504
f5f419de 1505@node Rpcalc Declarations
bfa74976
RS
1506@subsection Declarations for @code{rpcalc}
1507
1508Here are the C and Bison declarations for the reverse polish notation
1509calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1510
1511@example
72d2299c 1512/* Reverse polish notation calculator. */
bfa74976
RS
1513
1514%@{
38a92d50
PE
1515 #define YYSTYPE double
1516 #include <math.h>
1517 int yylex (void);
1518 void yyerror (char const *);
bfa74976
RS
1519%@}
1520
1521%token NUM
1522
72d2299c 1523%% /* Grammar rules and actions follow. */
bfa74976
RS
1524@end example
1525
75f5aaea 1526The declarations section (@pxref{Prologue, , The prologue}) contains two
38a92d50 1527preprocessor directives and two forward declarations.
bfa74976
RS
1528
1529The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1964ad8c
AD
1530specifying the C data type for semantic values of both tokens and
1531groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1532Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1533don't define it, @code{int} is the default. Because we specify
1534@code{double}, each token and each expression has an associated value,
1535which is a floating point number.
bfa74976
RS
1536
1537The @code{#include} directive is used to declare the exponentiation
1538function @code{pow}.
1539
38a92d50
PE
1540The forward declarations for @code{yylex} and @code{yyerror} are
1541needed because the C language requires that functions be declared
1542before they are used. These functions will be defined in the
1543epilogue, but the parser calls them so they must be declared in the
1544prologue.
1545
704a47c4
AD
1546The second section, Bison declarations, provides information to Bison
1547about the token types (@pxref{Bison Declarations, ,The Bison
1548Declarations Section}). Each terminal symbol that is not a
1549single-character literal must be declared here. (Single-character
bfa74976
RS
1550literals normally don't need to be declared.) In this example, all the
1551arithmetic operators are designated by single-character literals, so the
1552only terminal symbol that needs to be declared is @code{NUM}, the token
1553type for numeric constants.
1554
342b8b6e 1555@node Rpcalc Rules
bfa74976
RS
1556@subsection Grammar Rules for @code{rpcalc}
1557
1558Here are the grammar rules for the reverse polish notation calculator.
1559
1560@example
1561input: /* empty */
1562 | input line
1563;
1564
1565line: '\n'
18b519c0 1566 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
bfa74976
RS
1567;
1568
18b519c0
AD
1569exp: NUM @{ $$ = $1; @}
1570 | exp exp '+' @{ $$ = $1 + $2; @}
1571 | exp exp '-' @{ $$ = $1 - $2; @}
1572 | exp exp '*' @{ $$ = $1 * $2; @}
1573 | exp exp '/' @{ $$ = $1 / $2; @}
1574 /* Exponentiation */
1575 | exp exp '^' @{ $$ = pow ($1, $2); @}
1576 /* Unary minus */
1577 | exp 'n' @{ $$ = -$1; @}
bfa74976
RS
1578;
1579%%
1580@end example
1581
1582The groupings of the rpcalc ``language'' defined here are the expression
1583(given the name @code{exp}), the line of input (@code{line}), and the
1584complete input transcript (@code{input}). Each of these nonterminal
8c5b881d 1585symbols has several alternate rules, joined by the vertical bar @samp{|}
bfa74976
RS
1586which is read as ``or''. The following sections explain what these rules
1587mean.
1588
1589The semantics of the language is determined by the actions taken when a
1590grouping is recognized. The actions are the C code that appears inside
1591braces. @xref{Actions}.
1592
1593You must specify these actions in C, but Bison provides the means for
1594passing semantic values between the rules. In each action, the
1595pseudo-variable @code{$$} stands for the semantic value for the grouping
1596that the rule is going to construct. Assigning a value to @code{$$} is the
1597main job of most actions. The semantic values of the components of the
1598rule are referred to as @code{$1}, @code{$2}, and so on.
1599
1600@menu
13863333
AD
1601* Rpcalc Input::
1602* Rpcalc Line::
1603* Rpcalc Expr::
bfa74976
RS
1604@end menu
1605
342b8b6e 1606@node Rpcalc Input
bfa74976
RS
1607@subsubsection Explanation of @code{input}
1608
1609Consider the definition of @code{input}:
1610
1611@example
1612input: /* empty */
1613 | input line
1614;
1615@end example
1616
1617This definition reads as follows: ``A complete input is either an empty
1618string, or a complete input followed by an input line''. Notice that
1619``complete input'' is defined in terms of itself. This definition is said
1620to be @dfn{left recursive} since @code{input} appears always as the
1621leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1622
1623The first alternative is empty because there are no symbols between the
1624colon and the first @samp{|}; this means that @code{input} can match an
1625empty string of input (no tokens). We write the rules this way because it
1626is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1627It's conventional to put an empty alternative first and write the comment
1628@samp{/* empty */} in it.
1629
1630The second alternate rule (@code{input line}) handles all nontrivial input.
1631It means, ``After reading any number of lines, read one more line if
1632possible.'' The left recursion makes this rule into a loop. Since the
1633first alternative matches empty input, the loop can be executed zero or
1634more times.
1635
1636The parser function @code{yyparse} continues to process input until a
1637grammatical error is seen or the lexical analyzer says there are no more
72d2299c 1638input tokens; we will arrange for the latter to happen at end-of-input.
bfa74976 1639
342b8b6e 1640@node Rpcalc Line
bfa74976
RS
1641@subsubsection Explanation of @code{line}
1642
1643Now consider the definition of @code{line}:
1644
1645@example
1646line: '\n'
1647 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1648;
1649@end example
1650
1651The first alternative is a token which is a newline character; this means
1652that rpcalc accepts a blank line (and ignores it, since there is no
1653action). The second alternative is an expression followed by a newline.
1654This is the alternative that makes rpcalc useful. The semantic value of
1655the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1656question is the first symbol in the alternative. The action prints this
1657value, which is the result of the computation the user asked for.
1658
1659This action is unusual because it does not assign a value to @code{$$}. As
1660a consequence, the semantic value associated with the @code{line} is
1661uninitialized (its value will be unpredictable). This would be a bug if
1662that value were ever used, but we don't use it: once rpcalc has printed the
1663value of the user's input line, that value is no longer needed.
1664
342b8b6e 1665@node Rpcalc Expr
bfa74976
RS
1666@subsubsection Explanation of @code{expr}
1667
1668The @code{exp} grouping has several rules, one for each kind of expression.
1669The first rule handles the simplest expressions: those that are just numbers.
1670The second handles an addition-expression, which looks like two expressions
1671followed by a plus-sign. The third handles subtraction, and so on.
1672
1673@example
1674exp: NUM
1675 | exp exp '+' @{ $$ = $1 + $2; @}
1676 | exp exp '-' @{ $$ = $1 - $2; @}
1677 @dots{}
1678 ;
1679@end example
1680
1681We have used @samp{|} to join all the rules for @code{exp}, but we could
1682equally well have written them separately:
1683
1684@example
1685exp: NUM ;
1686exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1687exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1688 @dots{}
1689@end example
1690
1691Most of the rules have actions that compute the value of the expression in
1692terms of the value of its parts. For example, in the rule for addition,
1693@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1694the second one. The third component, @code{'+'}, has no meaningful
1695associated semantic value, but if it had one you could refer to it as
1696@code{$3}. When @code{yyparse} recognizes a sum expression using this
1697rule, the sum of the two subexpressions' values is produced as the value of
1698the entire expression. @xref{Actions}.
1699
1700You don't have to give an action for every rule. When a rule has no
1701action, Bison by default copies the value of @code{$1} into @code{$$}.
1702This is what happens in the first rule (the one that uses @code{NUM}).
1703
1704The formatting shown here is the recommended convention, but Bison does
72d2299c 1705not require it. You can add or change white space as much as you wish.
bfa74976
RS
1706For example, this:
1707
1708@example
99a9344e 1709exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
bfa74976
RS
1710@end example
1711
1712@noindent
1713means the same thing as this:
1714
1715@example
1716exp: NUM
1717 | exp exp '+' @{ $$ = $1 + $2; @}
1718 | @dots{}
99a9344e 1719;
bfa74976
RS
1720@end example
1721
1722@noindent
1723The latter, however, is much more readable.
1724
342b8b6e 1725@node Rpcalc Lexer
bfa74976
RS
1726@subsection The @code{rpcalc} Lexical Analyzer
1727@cindex writing a lexical analyzer
1728@cindex lexical analyzer, writing
1729
704a47c4
AD
1730The lexical analyzer's job is low-level parsing: converting characters
1731or sequences of characters into tokens. The Bison parser gets its
1732tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1733Analyzer Function @code{yylex}}.
bfa74976 1734
8a4281b9 1735Only a simple lexical analyzer is needed for the RPN
c827f760 1736calculator. This
bfa74976
RS
1737lexical analyzer skips blanks and tabs, then reads in numbers as
1738@code{double} and returns them as @code{NUM} tokens. Any other character
1739that isn't part of a number is a separate token. Note that the token-code
1740for such a single-character token is the character itself.
1741
1742The return value of the lexical analyzer function is a numeric code which
1743represents a token type. The same text used in Bison rules to stand for
1744this token type is also a C expression for the numeric code for the type.
1745This works in two ways. If the token type is a character literal, then its
e966383b 1746numeric code is that of the character; you can use the same
bfa74976
RS
1747character literal in the lexical analyzer to express the number. If the
1748token type is an identifier, that identifier is defined by Bison as a C
1749macro whose definition is the appropriate number. In this example,
1750therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1751
1964ad8c
AD
1752The semantic value of the token (if it has one) is stored into the
1753global variable @code{yylval}, which is where the Bison parser will look
1754for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
f5f419de 1755defined at the beginning of the grammar; @pxref{Rpcalc Declarations,
1964ad8c 1756,Declarations for @code{rpcalc}}.)
bfa74976 1757
72d2299c
PE
1758A token type code of zero is returned if the end-of-input is encountered.
1759(Bison recognizes any nonpositive value as indicating end-of-input.)
bfa74976
RS
1760
1761Here is the code for the lexical analyzer:
1762
1763@example
1764@group
72d2299c 1765/* The lexical analyzer returns a double floating point
e966383b 1766 number on the stack and the token NUM, or the numeric code
72d2299c
PE
1767 of the character read if not a number. It skips all blanks
1768 and tabs, and returns 0 for end-of-input. */
bfa74976
RS
1769
1770#include <ctype.h>
1771@end group
1772
1773@group
13863333
AD
1774int
1775yylex (void)
bfa74976
RS
1776@{
1777 int c;
1778
72d2299c 1779 /* Skip white space. */
13863333 1780 while ((c = getchar ()) == ' ' || c == '\t')
bfa74976
RS
1781 ;
1782@end group
1783@group
72d2299c 1784 /* Process numbers. */
13863333 1785 if (c == '.' || isdigit (c))
bfa74976
RS
1786 @{
1787 ungetc (c, stdin);
1788 scanf ("%lf", &yylval);
1789 return NUM;
1790 @}
1791@end group
1792@group
72d2299c 1793 /* Return end-of-input. */
13863333 1794 if (c == EOF)
bfa74976 1795 return 0;
72d2299c 1796 /* Return a single char. */
13863333 1797 return c;
bfa74976
RS
1798@}
1799@end group
1800@end example
1801
342b8b6e 1802@node Rpcalc Main
bfa74976
RS
1803@subsection The Controlling Function
1804@cindex controlling function
1805@cindex main function in simple example
1806
1807In keeping with the spirit of this example, the controlling function is
1808kept to the bare minimum. The only requirement is that it call
1809@code{yyparse} to start the process of parsing.
1810
1811@example
1812@group
13863333
AD
1813int
1814main (void)
bfa74976 1815@{
13863333 1816 return yyparse ();
bfa74976
RS
1817@}
1818@end group
1819@end example
1820
342b8b6e 1821@node Rpcalc Error
bfa74976
RS
1822@subsection The Error Reporting Routine
1823@cindex error reporting routine
1824
1825When @code{yyparse} detects a syntax error, it calls the error reporting
13863333 1826function @code{yyerror} to print an error message (usually but not
6e649e65 1827always @code{"syntax error"}). It is up to the programmer to supply
13863333
AD
1828@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1829here is the definition we will use:
bfa74976
RS
1830
1831@example
1832@group
1833#include <stdio.h>
1834
38a92d50 1835/* Called by yyparse on error. */
13863333 1836void
38a92d50 1837yyerror (char const *s)
bfa74976 1838@{
4e03e201 1839 fprintf (stderr, "%s\n", s);
bfa74976
RS
1840@}
1841@end group
1842@end example
1843
1844After @code{yyerror} returns, the Bison parser may recover from the error
1845and continue parsing if the grammar contains a suitable error rule
1846(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1847have not written any error rules in this example, so any invalid input will
1848cause the calculator program to exit. This is not clean behavior for a
9ecbd125 1849real calculator, but it is adequate for the first example.
bfa74976 1850
f5f419de 1851@node Rpcalc Generate
bfa74976
RS
1852@subsection Running Bison to Make the Parser
1853@cindex running Bison (introduction)
1854
ceed8467
AD
1855Before running Bison to produce a parser, we need to decide how to
1856arrange all the source code in one or more source files. For such a
ff7571c0
JD
1857simple example, the easiest thing is to put everything in one file,
1858the grammar file. The definitions of @code{yylex}, @code{yyerror} and
1859@code{main} go at the end, in the epilogue of the grammar file
75f5aaea 1860(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
bfa74976
RS
1861
1862For a large project, you would probably have several source files, and use
1863@code{make} to arrange to recompile them.
1864
ff7571c0
JD
1865With all the source in the grammar file, you use the following command
1866to convert it into a parser implementation file:
bfa74976
RS
1867
1868@example
fa4d969f 1869bison @var{file}.y
bfa74976
RS
1870@end example
1871
1872@noindent
ff7571c0
JD
1873In this example, the grammar file is called @file{rpcalc.y} (for
1874``Reverse Polish @sc{calc}ulator''). Bison produces a parser
1875implementation file named @file{@var{file}.tab.c}, removing the
1876@samp{.y} from the grammar file name. The parser implementation file
1877contains the source code for @code{yyparse}. The additional functions
1878in the grammar file (@code{yylex}, @code{yyerror} and @code{main}) are
1879copied verbatim to the parser implementation file.
bfa74976 1880
342b8b6e 1881@node Rpcalc Compile
ff7571c0 1882@subsection Compiling the Parser Implementation File
bfa74976
RS
1883@cindex compiling the parser
1884
ff7571c0 1885Here is how to compile and run the parser implementation file:
bfa74976
RS
1886
1887@example
1888@group
1889# @r{List files in current directory.}
9edcd895 1890$ @kbd{ls}
bfa74976
RS
1891rpcalc.tab.c rpcalc.y
1892@end group
1893
1894@group
1895# @r{Compile the Bison parser.}
1896# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
b56471a6 1897$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
bfa74976
RS
1898@end group
1899
1900@group
1901# @r{List files again.}
9edcd895 1902$ @kbd{ls}
bfa74976
RS
1903rpcalc rpcalc.tab.c rpcalc.y
1904@end group
1905@end example
1906
1907The file @file{rpcalc} now contains the executable code. Here is an
1908example session using @code{rpcalc}.
1909
1910@example
9edcd895
AD
1911$ @kbd{rpcalc}
1912@kbd{4 9 +}
bfa74976 191313
9edcd895 1914@kbd{3 7 + 3 4 5 *+-}
bfa74976 1915-13
9edcd895 1916@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
bfa74976 191713
9edcd895 1918@kbd{5 6 / 4 n +}
bfa74976 1919-3.166666667
9edcd895 1920@kbd{3 4 ^} @r{Exponentiation}
bfa74976 192181
9edcd895
AD
1922@kbd{^D} @r{End-of-file indicator}
1923$
bfa74976
RS
1924@end example
1925
342b8b6e 1926@node Infix Calc
bfa74976
RS
1927@section Infix Notation Calculator: @code{calc}
1928@cindex infix notation calculator
1929@cindex @code{calc}
1930@cindex calculator, infix notation
1931
1932We now modify rpcalc to handle infix operators instead of postfix. Infix
1933notation involves the concept of operator precedence and the need for
1934parentheses nested to arbitrary depth. Here is the Bison code for
1935@file{calc.y}, an infix desk-top calculator.
1936
1937@example
38a92d50 1938/* Infix notation calculator. */
bfa74976
RS
1939
1940%@{
38a92d50
PE
1941 #define YYSTYPE double
1942 #include <math.h>
1943 #include <stdio.h>
1944 int yylex (void);
1945 void yyerror (char const *);
bfa74976
RS
1946%@}
1947
38a92d50 1948/* Bison declarations. */
bfa74976
RS
1949%token NUM
1950%left '-' '+'
1951%left '*' '/'
d78f0ac9
AD
1952%precedence NEG /* negation--unary minus */
1953%right '^' /* exponentiation */
bfa74976 1954
38a92d50
PE
1955%% /* The grammar follows. */
1956input: /* empty */
bfa74976
RS
1957 | input line
1958;
1959
1960line: '\n'
1961 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1962;
1963
1964exp: NUM @{ $$ = $1; @}
1965 | exp '+' exp @{ $$ = $1 + $3; @}
1966 | exp '-' exp @{ $$ = $1 - $3; @}
1967 | exp '*' exp @{ $$ = $1 * $3; @}
1968 | exp '/' exp @{ $$ = $1 / $3; @}
1969 | '-' exp %prec NEG @{ $$ = -$2; @}
1970 | exp '^' exp @{ $$ = pow ($1, $3); @}
1971 | '(' exp ')' @{ $$ = $2; @}
1972;
1973%%
1974@end example
1975
1976@noindent
ceed8467
AD
1977The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1978same as before.
bfa74976
RS
1979
1980There are two important new features shown in this code.
1981
1982In the second section (Bison declarations), @code{%left} declares token
1983types and says they are left-associative operators. The declarations
1984@code{%left} and @code{%right} (right associativity) take the place of
1985@code{%token} which is used to declare a token type name without
d78f0ac9 1986associativity/precedence. (These tokens are single-character literals, which
bfa74976 1987ordinarily don't need to be declared. We declare them here to specify
d78f0ac9 1988the associativity/precedence.)
bfa74976
RS
1989
1990Operator precedence is determined by the line ordering of the
1991declarations; the higher the line number of the declaration (lower on
1992the page or screen), the higher the precedence. Hence, exponentiation
1993has the highest precedence, unary minus (@code{NEG}) is next, followed
d78f0ac9
AD
1994by @samp{*} and @samp{/}, and so on. Unary minus is not associative,
1995only precedence matters (@code{%precedence}. @xref{Precedence, ,Operator
704a47c4 1996Precedence}.
bfa74976 1997
704a47c4
AD
1998The other important new feature is the @code{%prec} in the grammar
1999section for the unary minus operator. The @code{%prec} simply instructs
2000Bison that the rule @samp{| '-' exp} has the same precedence as
2001@code{NEG}---in this case the next-to-highest. @xref{Contextual
2002Precedence, ,Context-Dependent Precedence}.
bfa74976
RS
2003
2004Here is a sample run of @file{calc.y}:
2005
2006@need 500
2007@example
9edcd895
AD
2008$ @kbd{calc}
2009@kbd{4 + 4.5 - (34/(8*3+-3))}
bfa74976 20106.880952381
9edcd895 2011@kbd{-56 + 2}
bfa74976 2012-54
9edcd895 2013@kbd{3 ^ 2}
bfa74976
RS
20149
2015@end example
2016
342b8b6e 2017@node Simple Error Recovery
bfa74976
RS
2018@section Simple Error Recovery
2019@cindex error recovery, simple
2020
2021Up to this point, this manual has not addressed the issue of @dfn{error
2022recovery}---how to continue parsing after the parser detects a syntax
ceed8467
AD
2023error. All we have handled is error reporting with @code{yyerror}.
2024Recall that by default @code{yyparse} returns after calling
2025@code{yyerror}. This means that an erroneous input line causes the
2026calculator program to exit. Now we show how to rectify this deficiency.
bfa74976
RS
2027
2028The Bison language itself includes the reserved word @code{error}, which
2029may be included in the grammar rules. In the example below it has
2030been added to one of the alternatives for @code{line}:
2031
2032@example
2033@group
2034line: '\n'
2035 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2036 | error '\n' @{ yyerrok; @}
2037;
2038@end group
2039@end example
2040
ceed8467 2041This addition to the grammar allows for simple error recovery in the
6e649e65 2042event of a syntax error. If an expression that cannot be evaluated is
ceed8467
AD
2043read, the error will be recognized by the third rule for @code{line},
2044and parsing will continue. (The @code{yyerror} function is still called
2045upon to print its message as well.) The action executes the statement
2046@code{yyerrok}, a macro defined automatically by Bison; its meaning is
2047that error recovery is complete (@pxref{Error Recovery}). Note the
2048difference between @code{yyerrok} and @code{yyerror}; neither one is a
e0c471a9 2049misprint.
bfa74976
RS
2050
2051This form of error recovery deals with syntax errors. There are other
2052kinds of errors; for example, division by zero, which raises an exception
2053signal that is normally fatal. A real calculator program must handle this
2054signal and use @code{longjmp} to return to @code{main} and resume parsing
2055input lines; it would also have to discard the rest of the current line of
2056input. We won't discuss this issue further because it is not specific to
2057Bison programs.
2058
342b8b6e
AD
2059@node Location Tracking Calc
2060@section Location Tracking Calculator: @code{ltcalc}
2061@cindex location tracking calculator
2062@cindex @code{ltcalc}
2063@cindex calculator, location tracking
2064
9edcd895
AD
2065This example extends the infix notation calculator with location
2066tracking. This feature will be used to improve the error messages. For
2067the sake of clarity, this example is a simple integer calculator, since
2068most of the work needed to use locations will be done in the lexical
72d2299c 2069analyzer.
342b8b6e
AD
2070
2071@menu
f5f419de
DJ
2072* Ltcalc Declarations:: Bison and C declarations for ltcalc.
2073* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
2074* Ltcalc Lexer:: The lexical analyzer.
342b8b6e
AD
2075@end menu
2076
f5f419de 2077@node Ltcalc Declarations
342b8b6e
AD
2078@subsection Declarations for @code{ltcalc}
2079
9edcd895
AD
2080The C and Bison declarations for the location tracking calculator are
2081the same as the declarations for the infix notation calculator.
342b8b6e
AD
2082
2083@example
2084/* Location tracking calculator. */
2085
2086%@{
38a92d50
PE
2087 #define YYSTYPE int
2088 #include <math.h>
2089 int yylex (void);
2090 void yyerror (char const *);
342b8b6e
AD
2091%@}
2092
2093/* Bison declarations. */
2094%token NUM
2095
2096%left '-' '+'
2097%left '*' '/'
d78f0ac9 2098%precedence NEG
342b8b6e
AD
2099%right '^'
2100
38a92d50 2101%% /* The grammar follows. */
342b8b6e
AD
2102@end example
2103
9edcd895
AD
2104@noindent
2105Note there are no declarations specific to locations. Defining a data
2106type for storing locations is not needed: we will use the type provided
2107by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2108four member structure with the following integer fields:
2109@code{first_line}, @code{first_column}, @code{last_line} and
cd48d21d
AD
2110@code{last_column}. By conventions, and in accordance with the GNU
2111Coding Standards and common practice, the line and column count both
2112start at 1.
342b8b6e
AD
2113
2114@node Ltcalc Rules
2115@subsection Grammar Rules for @code{ltcalc}
2116
9edcd895
AD
2117Whether handling locations or not has no effect on the syntax of your
2118language. Therefore, grammar rules for this example will be very close
2119to those of the previous example: we will only modify them to benefit
2120from the new information.
342b8b6e 2121
9edcd895
AD
2122Here, we will use locations to report divisions by zero, and locate the
2123wrong expressions or subexpressions.
342b8b6e
AD
2124
2125@example
2126@group
2127input : /* empty */
2128 | input line
2129;
2130@end group
2131
2132@group
2133line : '\n'
2134 | exp '\n' @{ printf ("%d\n", $1); @}
2135;
2136@end group
2137
2138@group
2139exp : NUM @{ $$ = $1; @}
2140 | exp '+' exp @{ $$ = $1 + $3; @}
2141 | exp '-' exp @{ $$ = $1 - $3; @}
2142 | exp '*' exp @{ $$ = $1 * $3; @}
2143@end group
342b8b6e 2144@group
9edcd895 2145 | exp '/' exp
342b8b6e
AD
2146 @{
2147 if ($3)
2148 $$ = $1 / $3;
2149 else
2150 @{
2151 $$ = 1;
9edcd895
AD
2152 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2153 @@3.first_line, @@3.first_column,
2154 @@3.last_line, @@3.last_column);
342b8b6e
AD
2155 @}
2156 @}
2157@end group
2158@group
178e123e 2159 | '-' exp %prec NEG @{ $$ = -$2; @}
342b8b6e
AD
2160 | exp '^' exp @{ $$ = pow ($1, $3); @}
2161 | '(' exp ')' @{ $$ = $2; @}
2162@end group
2163@end example
2164
2165This code shows how to reach locations inside of semantic actions, by
2166using the pseudo-variables @code{@@@var{n}} for rule components, and the
2167pseudo-variable @code{@@$} for groupings.
2168
9edcd895
AD
2169We don't need to assign a value to @code{@@$}: the output parser does it
2170automatically. By default, before executing the C code of each action,
2171@code{@@$} is set to range from the beginning of @code{@@1} to the end
2172of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2173can be redefined (@pxref{Location Default Action, , Default Action for
2174Locations}), and for very specific rules, @code{@@$} can be computed by
2175hand.
342b8b6e
AD
2176
2177@node Ltcalc Lexer
2178@subsection The @code{ltcalc} Lexical Analyzer.
2179
9edcd895 2180Until now, we relied on Bison's defaults to enable location
72d2299c 2181tracking. The next step is to rewrite the lexical analyzer, and make it
9edcd895
AD
2182able to feed the parser with the token locations, as it already does for
2183semantic values.
342b8b6e 2184
9edcd895
AD
2185To this end, we must take into account every single character of the
2186input text, to avoid the computed locations of being fuzzy or wrong:
342b8b6e
AD
2187
2188@example
2189@group
2190int
2191yylex (void)
2192@{
2193 int c;
18b519c0 2194@end group
342b8b6e 2195
18b519c0 2196@group
72d2299c 2197 /* Skip white space. */
342b8b6e
AD
2198 while ((c = getchar ()) == ' ' || c == '\t')
2199 ++yylloc.last_column;
18b519c0 2200@end group
342b8b6e 2201
18b519c0 2202@group
72d2299c 2203 /* Step. */
342b8b6e
AD
2204 yylloc.first_line = yylloc.last_line;
2205 yylloc.first_column = yylloc.last_column;
2206@end group
2207
2208@group
72d2299c 2209 /* Process numbers. */
342b8b6e
AD
2210 if (isdigit (c))
2211 @{
2212 yylval = c - '0';
2213 ++yylloc.last_column;
2214 while (isdigit (c = getchar ()))
2215 @{
2216 ++yylloc.last_column;
2217 yylval = yylval * 10 + c - '0';
2218 @}
2219 ungetc (c, stdin);
2220 return NUM;
2221 @}
2222@end group
2223
72d2299c 2224 /* Return end-of-input. */
342b8b6e
AD
2225 if (c == EOF)
2226 return 0;
2227
72d2299c 2228 /* Return a single char, and update location. */
342b8b6e
AD
2229 if (c == '\n')
2230 @{
2231 ++yylloc.last_line;
2232 yylloc.last_column = 0;
2233 @}
2234 else
2235 ++yylloc.last_column;
2236 return c;
2237@}
2238@end example
2239
9edcd895
AD
2240Basically, the lexical analyzer performs the same processing as before:
2241it skips blanks and tabs, and reads numbers or single-character tokens.
2242In addition, it updates @code{yylloc}, the global variable (of type
2243@code{YYLTYPE}) containing the token's location.
342b8b6e 2244
9edcd895 2245Now, each time this function returns a token, the parser has its number
72d2299c 2246as well as its semantic value, and its location in the text. The last
9edcd895
AD
2247needed change is to initialize @code{yylloc}, for example in the
2248controlling function:
342b8b6e
AD
2249
2250@example
9edcd895 2251@group
342b8b6e
AD
2252int
2253main (void)
2254@{
2255 yylloc.first_line = yylloc.last_line = 1;
2256 yylloc.first_column = yylloc.last_column = 0;
2257 return yyparse ();
2258@}
9edcd895 2259@end group
342b8b6e
AD
2260@end example
2261
9edcd895
AD
2262Remember that computing locations is not a matter of syntax. Every
2263character must be associated to a location update, whether it is in
2264valid input, in comments, in literal strings, and so on.
342b8b6e
AD
2265
2266@node Multi-function Calc
bfa74976
RS
2267@section Multi-Function Calculator: @code{mfcalc}
2268@cindex multi-function calculator
2269@cindex @code{mfcalc}
2270@cindex calculator, multi-function
2271
2272Now that the basics of Bison have been discussed, it is time to move on to
2273a more advanced problem. The above calculators provided only five
2274functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2275be nice to have a calculator that provides other mathematical functions such
2276as @code{sin}, @code{cos}, etc.
2277
2278It is easy to add new operators to the infix calculator as long as they are
2279only single-character literals. The lexical analyzer @code{yylex} passes
9d9b8b70 2280back all nonnumeric characters as tokens, so new grammar rules suffice for
bfa74976
RS
2281adding a new operator. But we want something more flexible: built-in
2282functions whose syntax has this form:
2283
2284@example
2285@var{function_name} (@var{argument})
2286@end example
2287
2288@noindent
2289At the same time, we will add memory to the calculator, by allowing you
2290to create named variables, store values in them, and use them later.
2291Here is a sample session with the multi-function calculator:
2292
2293@example
9edcd895
AD
2294$ @kbd{mfcalc}
2295@kbd{pi = 3.141592653589}
bfa74976 22963.1415926536
9edcd895 2297@kbd{sin(pi)}
bfa74976 22980.0000000000
9edcd895 2299@kbd{alpha = beta1 = 2.3}
bfa74976 23002.3000000000
9edcd895 2301@kbd{alpha}
bfa74976 23022.3000000000
9edcd895 2303@kbd{ln(alpha)}
bfa74976 23040.8329091229
9edcd895 2305@kbd{exp(ln(beta1))}
bfa74976 23062.3000000000
9edcd895 2307$
bfa74976
RS
2308@end example
2309
2310Note that multiple assignment and nested function calls are permitted.
2311
2312@menu
f5f419de
DJ
2313* Mfcalc Declarations:: Bison declarations for multi-function calculator.
2314* Mfcalc Rules:: Grammar rules for the calculator.
2315* Mfcalc Symbol Table:: Symbol table management subroutines.
bfa74976
RS
2316@end menu
2317
f5f419de 2318@node Mfcalc Declarations
bfa74976
RS
2319@subsection Declarations for @code{mfcalc}
2320
2321Here are the C and Bison declarations for the multi-function calculator.
2322
2323@smallexample
18b519c0 2324@group
bfa74976 2325%@{
38a92d50
PE
2326 #include <math.h> /* For math functions, cos(), sin(), etc. */
2327 #include "calc.h" /* Contains definition of `symrec'. */
2328 int yylex (void);
2329 void yyerror (char const *);
bfa74976 2330%@}
18b519c0
AD
2331@end group
2332@group
bfa74976 2333%union @{
38a92d50
PE
2334 double val; /* For returning numbers. */
2335 symrec *tptr; /* For returning symbol-table pointers. */
bfa74976 2336@}
18b519c0 2337@end group
38a92d50
PE
2338%token <val> NUM /* Simple double precision number. */
2339%token <tptr> VAR FNCT /* Variable and Function. */
bfa74976
RS
2340%type <val> exp
2341
18b519c0 2342@group
bfa74976
RS
2343%right '='
2344%left '-' '+'
2345%left '*' '/'
d78f0ac9
AD
2346%precedence NEG /* negation--unary minus */
2347%right '^' /* exponentiation */
18b519c0 2348@end group
38a92d50 2349%% /* The grammar follows. */
bfa74976
RS
2350@end smallexample
2351
2352The above grammar introduces only two new features of the Bison language.
2353These features allow semantic values to have various data types
2354(@pxref{Multiple Types, ,More Than One Value Type}).
2355
2356The @code{%union} declaration specifies the entire list of possible types;
2357this is instead of defining @code{YYSTYPE}. The allowable types are now
2358double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2359the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2360
2361Since values can now have various types, it is necessary to associate a
2362type with each grammar symbol whose semantic value is used. These symbols
2363are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2364declarations are augmented with information about their data type (placed
2365between angle brackets).
2366
704a47c4
AD
2367The Bison construct @code{%type} is used for declaring nonterminal
2368symbols, just as @code{%token} is used for declaring token types. We
2369have not used @code{%type} before because nonterminal symbols are
2370normally declared implicitly by the rules that define them. But
2371@code{exp} must be declared explicitly so we can specify its value type.
2372@xref{Type Decl, ,Nonterminal Symbols}.
bfa74976 2373
342b8b6e 2374@node Mfcalc Rules
bfa74976
RS
2375@subsection Grammar Rules for @code{mfcalc}
2376
2377Here are the grammar rules for the multi-function calculator.
2378Most of them are copied directly from @code{calc}; three rules,
2379those which mention @code{VAR} or @code{FNCT}, are new.
2380
2381@smallexample
18b519c0 2382@group
bfa74976
RS
2383input: /* empty */
2384 | input line
2385;
18b519c0 2386@end group
bfa74976 2387
18b519c0 2388@group
bfa74976
RS
2389line:
2390 '\n'
2391 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2392 | error '\n' @{ yyerrok; @}
2393;
18b519c0 2394@end group
bfa74976 2395
18b519c0 2396@group
bfa74976
RS
2397exp: NUM @{ $$ = $1; @}
2398 | VAR @{ $$ = $1->value.var; @}
2399 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2400 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2401 | exp '+' exp @{ $$ = $1 + $3; @}
2402 | exp '-' exp @{ $$ = $1 - $3; @}
2403 | exp '*' exp @{ $$ = $1 * $3; @}
2404 | exp '/' exp @{ $$ = $1 / $3; @}
2405 | '-' exp %prec NEG @{ $$ = -$2; @}
2406 | exp '^' exp @{ $$ = pow ($1, $3); @}
2407 | '(' exp ')' @{ $$ = $2; @}
2408;
18b519c0 2409@end group
38a92d50 2410/* End of grammar. */
bfa74976
RS
2411%%
2412@end smallexample
2413
f5f419de 2414@node Mfcalc Symbol Table
bfa74976
RS
2415@subsection The @code{mfcalc} Symbol Table
2416@cindex symbol table example
2417
2418The multi-function calculator requires a symbol table to keep track of the
2419names and meanings of variables and functions. This doesn't affect the
2420grammar rules (except for the actions) or the Bison declarations, but it
2421requires some additional C functions for support.
2422
2423The symbol table itself consists of a linked list of records. Its
2424definition, which is kept in the header @file{calc.h}, is as follows. It
2425provides for either functions or variables to be placed in the table.
2426
2427@smallexample
2428@group
38a92d50 2429/* Function type. */
32dfccf8 2430typedef double (*func_t) (double);
72f889cc 2431@end group
32dfccf8 2432
72f889cc 2433@group
38a92d50 2434/* Data type for links in the chain of symbols. */
bfa74976
RS
2435struct symrec
2436@{
38a92d50 2437 char *name; /* name of symbol */
bfa74976 2438 int type; /* type of symbol: either VAR or FNCT */
32dfccf8
AD
2439 union
2440 @{
38a92d50
PE
2441 double var; /* value of a VAR */
2442 func_t fnctptr; /* value of a FNCT */
bfa74976 2443 @} value;
38a92d50 2444 struct symrec *next; /* link field */
bfa74976
RS
2445@};
2446@end group
2447
2448@group
2449typedef struct symrec symrec;
2450
38a92d50 2451/* The symbol table: a chain of `struct symrec'. */
bfa74976
RS
2452extern symrec *sym_table;
2453
a730d142 2454symrec *putsym (char const *, int);
38a92d50 2455symrec *getsym (char const *);
bfa74976
RS
2456@end group
2457@end smallexample
2458
2459The new version of @code{main} includes a call to @code{init_table}, a
2460function that initializes the symbol table. Here it is, and
2461@code{init_table} as well:
2462
2463@smallexample
bfa74976
RS
2464#include <stdio.h>
2465
18b519c0 2466@group
38a92d50 2467/* Called by yyparse on error. */
13863333 2468void
38a92d50 2469yyerror (char const *s)
bfa74976
RS
2470@{
2471 printf ("%s\n", s);
2472@}
18b519c0 2473@end group
bfa74976 2474
18b519c0 2475@group
bfa74976
RS
2476struct init
2477@{
38a92d50
PE
2478 char const *fname;
2479 double (*fnct) (double);
bfa74976
RS
2480@};
2481@end group
2482
2483@group
38a92d50 2484struct init const arith_fncts[] =
13863333 2485@{
32dfccf8
AD
2486 "sin", sin,
2487 "cos", cos,
13863333 2488 "atan", atan,
32dfccf8
AD
2489 "ln", log,
2490 "exp", exp,
13863333
AD
2491 "sqrt", sqrt,
2492 0, 0
2493@};
18b519c0 2494@end group
bfa74976 2495
18b519c0 2496@group
bfa74976 2497/* The symbol table: a chain of `struct symrec'. */
38a92d50 2498symrec *sym_table;
bfa74976
RS
2499@end group
2500
2501@group
72d2299c 2502/* Put arithmetic functions in table. */
13863333
AD
2503void
2504init_table (void)
bfa74976
RS
2505@{
2506 int i;
2507 symrec *ptr;
2508 for (i = 0; arith_fncts[i].fname != 0; i++)
2509 @{
2510 ptr = putsym (arith_fncts[i].fname, FNCT);
2511 ptr->value.fnctptr = arith_fncts[i].fnct;
2512 @}
2513@}
2514@end group
38a92d50
PE
2515
2516@group
2517int
2518main (void)
2519@{
2520 init_table ();
2521 return yyparse ();
2522@}
2523@end group
bfa74976
RS
2524@end smallexample
2525
2526By simply editing the initialization list and adding the necessary include
2527files, you can add additional functions to the calculator.
2528
2529Two important functions allow look-up and installation of symbols in the
2530symbol table. The function @code{putsym} is passed a name and the type
2531(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2532linked to the front of the list, and a pointer to the object is returned.
2533The function @code{getsym} is passed the name of the symbol to look up. If
2534found, a pointer to that symbol is returned; otherwise zero is returned.
2535
2536@smallexample
2537symrec *
38a92d50 2538putsym (char const *sym_name, int sym_type)
bfa74976
RS
2539@{
2540 symrec *ptr;
2541 ptr = (symrec *) malloc (sizeof (symrec));
2542 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2543 strcpy (ptr->name,sym_name);
2544 ptr->type = sym_type;
72d2299c 2545 ptr->value.var = 0; /* Set value to 0 even if fctn. */
bfa74976
RS
2546 ptr->next = (struct symrec *)sym_table;
2547 sym_table = ptr;
2548 return ptr;
2549@}
2550
2551symrec *
38a92d50 2552getsym (char const *sym_name)
bfa74976
RS
2553@{
2554 symrec *ptr;
2555 for (ptr = sym_table; ptr != (symrec *) 0;
2556 ptr = (symrec *)ptr->next)
2557 if (strcmp (ptr->name,sym_name) == 0)
2558 return ptr;
2559 return 0;
2560@}
2561@end smallexample
2562
2563The function @code{yylex} must now recognize variables, numeric values, and
2564the single-character arithmetic operators. Strings of alphanumeric
9d9b8b70 2565characters with a leading letter are recognized as either variables or
bfa74976
RS
2566functions depending on what the symbol table says about them.
2567
2568The string is passed to @code{getsym} for look up in the symbol table. If
2569the name appears in the table, a pointer to its location and its type
2570(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2571already in the table, then it is installed as a @code{VAR} using
2572@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
e0c471a9 2573returned to @code{yyparse}.
bfa74976
RS
2574
2575No change is needed in the handling of numeric values and arithmetic
2576operators in @code{yylex}.
2577
2578@smallexample
2579@group
2580#include <ctype.h>
18b519c0 2581@end group
13863333 2582
18b519c0 2583@group
13863333
AD
2584int
2585yylex (void)
bfa74976
RS
2586@{
2587 int c;
2588
72d2299c 2589 /* Ignore white space, get first nonwhite character. */
bfa74976
RS
2590 while ((c = getchar ()) == ' ' || c == '\t');
2591
2592 if (c == EOF)
2593 return 0;
2594@end group
2595
2596@group
2597 /* Char starts a number => parse the number. */
2598 if (c == '.' || isdigit (c))
2599 @{
2600 ungetc (c, stdin);
2601 scanf ("%lf", &yylval.val);
2602 return NUM;
2603 @}
2604@end group
2605
2606@group
2607 /* Char starts an identifier => read the name. */
2608 if (isalpha (c))
2609 @{
2610 symrec *s;
2611 static char *symbuf = 0;
2612 static int length = 0;
2613 int i;
2614@end group
2615
2616@group
2617 /* Initially make the buffer long enough
2618 for a 40-character symbol name. */
2619 if (length == 0)
2620 length = 40, symbuf = (char *)malloc (length + 1);
2621
2622 i = 0;
2623 do
2624@end group
2625@group
2626 @{
2627 /* If buffer is full, make it bigger. */
2628 if (i == length)
2629 @{
2630 length *= 2;
18b519c0 2631 symbuf = (char *) realloc (symbuf, length + 1);
bfa74976
RS
2632 @}
2633 /* Add this character to the buffer. */
2634 symbuf[i++] = c;
2635 /* Get another character. */
2636 c = getchar ();
2637 @}
2638@end group
2639@group
72d2299c 2640 while (isalnum (c));
bfa74976
RS
2641
2642 ungetc (c, stdin);
2643 symbuf[i] = '\0';
2644@end group
2645
2646@group
2647 s = getsym (symbuf);
2648 if (s == 0)
2649 s = putsym (symbuf, VAR);
2650 yylval.tptr = s;
2651 return s->type;
2652 @}
2653
2654 /* Any other character is a token by itself. */
2655 return c;
2656@}
2657@end group
2658@end smallexample
2659
72d2299c 2660This program is both powerful and flexible. You may easily add new
704a47c4
AD
2661functions, and it is a simple job to modify this code to install
2662predefined variables such as @code{pi} or @code{e} as well.
bfa74976 2663
342b8b6e 2664@node Exercises
bfa74976
RS
2665@section Exercises
2666@cindex exercises
2667
2668@enumerate
2669@item
2670Add some new functions from @file{math.h} to the initialization list.
2671
2672@item
2673Add another array that contains constants and their values. Then
2674modify @code{init_table} to add these constants to the symbol table.
2675It will be easiest to give the constants type @code{VAR}.
2676
2677@item
2678Make the program report an error if the user refers to an
2679uninitialized variable in any way except to store a value in it.
2680@end enumerate
2681
342b8b6e 2682@node Grammar File
bfa74976
RS
2683@chapter Bison Grammar Files
2684
2685Bison takes as input a context-free grammar specification and produces a
2686C-language function that recognizes correct instances of the grammar.
2687
ff7571c0 2688The Bison grammar file conventionally has a name ending in @samp{.y}.
234a3be3 2689@xref{Invocation, ,Invoking Bison}.
bfa74976
RS
2690
2691@menu
2692* Grammar Outline:: Overall layout of the grammar file.
2693* Symbols:: Terminal and nonterminal symbols.
2694* Rules:: How to write grammar rules.
2695* Recursion:: Writing recursive rules.
2696* Semantics:: Semantic values and actions.
847bf1f5 2697* Locations:: Locations and actions.
bfa74976
RS
2698* Declarations:: All kinds of Bison declarations are described here.
2699* Multiple Parsers:: Putting more than one Bison parser in one program.
2700@end menu
2701
342b8b6e 2702@node Grammar Outline
bfa74976
RS
2703@section Outline of a Bison Grammar
2704
2705A Bison grammar file has four main sections, shown here with the
2706appropriate delimiters:
2707
2708@example
2709%@{
38a92d50 2710 @var{Prologue}
bfa74976
RS
2711%@}
2712
2713@var{Bison declarations}
2714
2715%%
2716@var{Grammar rules}
2717%%
2718
75f5aaea 2719@var{Epilogue}
bfa74976
RS
2720@end example
2721
2722Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
8a4281b9 2723As a GNU extension, @samp{//} introduces a comment that
2bfc2e2a 2724continues until end of line.
bfa74976
RS
2725
2726@menu
f5f419de 2727* Prologue:: Syntax and usage of the prologue.
2cbe6b7f 2728* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
f5f419de
DJ
2729* Bison Declarations:: Syntax and usage of the Bison declarations section.
2730* Grammar Rules:: Syntax and usage of the grammar rules section.
2731* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
2732@end menu
2733
38a92d50 2734@node Prologue
75f5aaea
MA
2735@subsection The prologue
2736@cindex declarations section
2737@cindex Prologue
2738@cindex declarations
bfa74976 2739
f8e1c9e5
AD
2740The @var{Prologue} section contains macro definitions and declarations
2741of functions and variables that are used in the actions in the grammar
ff7571c0
JD
2742rules. These are copied to the beginning of the parser implementation
2743file so that they precede the definition of @code{yyparse}. You can
2744use @samp{#include} to get the declarations from a header file. If
2745you don't need any C declarations, you may omit the @samp{%@{} and
f8e1c9e5 2746@samp{%@}} delimiters that bracket this section.
bfa74976 2747
9c437126 2748The @var{Prologue} section is terminated by the first occurrence
287c78f6
PE
2749of @samp{%@}} that is outside a comment, a string literal, or a
2750character constant.
2751
c732d2c6
AD
2752You may have more than one @var{Prologue} section, intermixed with the
2753@var{Bison declarations}. This allows you to have C and Bison
2754declarations that refer to each other. For example, the @code{%union}
2755declaration may use types defined in a header file, and you may wish to
2756prototype functions that take arguments of type @code{YYSTYPE}. This
2757can be done with two @var{Prologue} blocks, one before and one after the
2758@code{%union} declaration.
2759
2760@smallexample
2761%@{
aef3da86 2762 #define _GNU_SOURCE
38a92d50
PE
2763 #include <stdio.h>
2764 #include "ptypes.h"
c732d2c6
AD
2765%@}
2766
2767%union @{
779e7ceb 2768 long int n;
c732d2c6
AD
2769 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2770@}
2771
2772%@{
38a92d50
PE
2773 static void print_token_value (FILE *, int, YYSTYPE);
2774 #define YYPRINT(F, N, L) print_token_value (F, N, L)
c732d2c6
AD
2775%@}
2776
2777@dots{}
2778@end smallexample
2779
aef3da86
PE
2780When in doubt, it is usually safer to put prologue code before all
2781Bison declarations, rather than after. For example, any definitions
2782of feature test macros like @code{_GNU_SOURCE} or
2783@code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2784feature test macros can affect the behavior of Bison-generated
2785@code{#include} directives.
2786
2cbe6b7f
JD
2787@node Prologue Alternatives
2788@subsection Prologue Alternatives
2789@cindex Prologue Alternatives
2790
136a0f76 2791@findex %code
16dc6a9e
JD
2792@findex %code requires
2793@findex %code provides
2794@findex %code top
85894313 2795
2cbe6b7f 2796The functionality of @var{Prologue} sections can often be subtle and
ff7571c0
JD
2797inflexible. As an alternative, Bison provides a @code{%code}
2798directive with an explicit qualifier field, which identifies the
2799purpose of the code and thus the location(s) where Bison should
2800generate it. For C/C++, the qualifier can be omitted for the default
2801location, or it can be one of @code{requires}, @code{provides},
e0c07222 2802@code{top}. @xref{%code Summary}.
2cbe6b7f
JD
2803
2804Look again at the example of the previous section:
2805
2806@smallexample
2807%@{
2808 #define _GNU_SOURCE
2809 #include <stdio.h>
2810 #include "ptypes.h"
2811%@}
2812
2813%union @{
2814 long int n;
2815 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2816@}
2817
2818%@{
2819 static void print_token_value (FILE *, int, YYSTYPE);
2820 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2821%@}
2822
2823@dots{}
2824@end smallexample
2825
2826@noindent
ff7571c0
JD
2827Notice that there are two @var{Prologue} sections here, but there's a
2828subtle distinction between their functionality. For example, if you
2829decide to override Bison's default definition for @code{YYLTYPE}, in
2830which @var{Prologue} section should you write your new definition?
2831You should write it in the first since Bison will insert that code
2832into the parser implementation file @emph{before} the default
2833@code{YYLTYPE} definition. In which @var{Prologue} section should you
2834prototype an internal function, @code{trace_token}, that accepts
2835@code{YYLTYPE} and @code{yytokentype} as arguments? You should
2836prototype it in the second since Bison will insert that code
2cbe6b7f
JD
2837@emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2838
2839This distinction in functionality between the two @var{Prologue} sections is
2840established by the appearance of the @code{%union} between them.
a501eca9 2841This behavior raises a few questions.
2cbe6b7f
JD
2842First, why should the position of a @code{%union} affect definitions related to
2843@code{YYLTYPE} and @code{yytokentype}?
2844Second, what if there is no @code{%union}?
2845In that case, the second kind of @var{Prologue} section is not available.
2846This behavior is not intuitive.
2847
8e0a5e9e 2848To avoid this subtle @code{%union} dependency, rewrite the example using a
16dc6a9e 2849@code{%code top} and an unqualified @code{%code}.
2cbe6b7f
JD
2850Let's go ahead and add the new @code{YYLTYPE} definition and the
2851@code{trace_token} prototype at the same time:
2852
2853@smallexample
16dc6a9e 2854%code top @{
2cbe6b7f
JD
2855 #define _GNU_SOURCE
2856 #include <stdio.h>
8e0a5e9e
JD
2857
2858 /* WARNING: The following code really belongs
16dc6a9e 2859 * in a `%code requires'; see below. */
8e0a5e9e 2860
2cbe6b7f
JD
2861 #include "ptypes.h"
2862 #define YYLTYPE YYLTYPE
2863 typedef struct YYLTYPE
2864 @{
2865 int first_line;
2866 int first_column;
2867 int last_line;
2868 int last_column;
2869 char *filename;
2870 @} YYLTYPE;
2871@}
2872
2873%union @{
2874 long int n;
2875 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2876@}
2877
2878%code @{
2879 static void print_token_value (FILE *, int, YYSTYPE);
2880 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2881 static void trace_token (enum yytokentype token, YYLTYPE loc);
2882@}
2883
2884@dots{}
2885@end smallexample
2886
2887@noindent
16dc6a9e
JD
2888In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2889functionality as the two kinds of @var{Prologue} sections, but it's always
8e0a5e9e 2890explicit which kind you intend.
2cbe6b7f
JD
2891Moreover, both kinds are always available even in the absence of @code{%union}.
2892
ff7571c0
JD
2893The @code{%code top} block above logically contains two parts. The
2894first two lines before the warning need to appear near the top of the
2895parser implementation file. The first line after the warning is
2896required by @code{YYSTYPE} and thus also needs to appear in the parser
2897implementation file. However, if you've instructed Bison to generate
2898a parser header file (@pxref{Decl Summary, ,%defines}), you probably
2899want that line to appear before the @code{YYSTYPE} definition in that
2900header file as well. The @code{YYLTYPE} definition should also appear
2901in the parser header file to override the default @code{YYLTYPE}
2902definition there.
2cbe6b7f 2903
16dc6a9e 2904In other words, in the @code{%code top} block above, all but the first two
8e0a5e9e
JD
2905lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2906definitions.
16dc6a9e 2907Thus, they belong in one or more @code{%code requires}:
9bc0dd67
JD
2908
2909@smallexample
16dc6a9e 2910%code top @{
2cbe6b7f
JD
2911 #define _GNU_SOURCE
2912 #include <stdio.h>
2913@}
2914
16dc6a9e 2915%code requires @{
9bc0dd67
JD
2916 #include "ptypes.h"
2917@}
2918%union @{
2919 long int n;
2920 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2921@}
2922
16dc6a9e 2923%code requires @{
2cbe6b7f
JD
2924 #define YYLTYPE YYLTYPE
2925 typedef struct YYLTYPE
2926 @{
2927 int first_line;
2928 int first_column;
2929 int last_line;
2930 int last_column;
2931 char *filename;
2932 @} YYLTYPE;
2933@}
2934
136a0f76 2935%code @{
2cbe6b7f
JD
2936 static void print_token_value (FILE *, int, YYSTYPE);
2937 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2938 static void trace_token (enum yytokentype token, YYLTYPE loc);
2939@}
2940
2941@dots{}
2942@end smallexample
2943
2944@noindent
ff7571c0
JD
2945Now Bison will insert @code{#include "ptypes.h"} and the new
2946@code{YYLTYPE} definition before the Bison-generated @code{YYSTYPE}
2947and @code{YYLTYPE} definitions in both the parser implementation file
2948and the parser header file. (By the same reasoning, @code{%code
2949requires} would also be the appropriate place to write your own
2950definition for @code{YYSTYPE}.)
2951
2952When you are writing dependency code for @code{YYSTYPE} and
2953@code{YYLTYPE}, you should prefer @code{%code requires} over
2954@code{%code top} regardless of whether you instruct Bison to generate
2955a parser header file. When you are writing code that you need Bison
2956to insert only into the parser implementation file and that has no
2957special need to appear at the top of that file, you should prefer the
2958unqualified @code{%code} over @code{%code top}. These practices will
2959make the purpose of each block of your code explicit to Bison and to
2960other developers reading your grammar file. Following these
2961practices, we expect the unqualified @code{%code} and @code{%code
2962requires} to be the most important of the four @var{Prologue}
16dc6a9e 2963alternatives.
a501eca9 2964
ff7571c0
JD
2965At some point while developing your parser, you might decide to
2966provide @code{trace_token} to modules that are external to your
2967parser. Thus, you might wish for Bison to insert the prototype into
2968both the parser header file and the parser implementation file. Since
2969this function is not a dependency required by @code{YYSTYPE} or
8e0a5e9e 2970@code{YYLTYPE}, it doesn't make sense to move its prototype to a
ff7571c0
JD
2971@code{%code requires}. More importantly, since it depends upon
2972@code{YYLTYPE} and @code{yytokentype}, @code{%code requires} is not
2973sufficient. Instead, move its prototype from the unqualified
2974@code{%code} to a @code{%code provides}:
2cbe6b7f
JD
2975
2976@smallexample
16dc6a9e 2977%code top @{
2cbe6b7f 2978 #define _GNU_SOURCE
136a0f76 2979 #include <stdio.h>
2cbe6b7f 2980@}
136a0f76 2981
16dc6a9e 2982%code requires @{
2cbe6b7f
JD
2983 #include "ptypes.h"
2984@}
2985%union @{
2986 long int n;
2987 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2988@}
2989
16dc6a9e 2990%code requires @{
2cbe6b7f
JD
2991 #define YYLTYPE YYLTYPE
2992 typedef struct YYLTYPE
2993 @{
2994 int first_line;
2995 int first_column;
2996 int last_line;
2997 int last_column;
2998 char *filename;
2999 @} YYLTYPE;
3000@}
3001
16dc6a9e 3002%code provides @{
2cbe6b7f
JD
3003 void trace_token (enum yytokentype token, YYLTYPE loc);
3004@}
3005
3006%code @{
9bc0dd67
JD
3007 static void print_token_value (FILE *, int, YYSTYPE);
3008 #define YYPRINT(F, N, L) print_token_value (F, N, L)
34f98f46 3009@}
9bc0dd67
JD
3010
3011@dots{}
3012@end smallexample
3013
2cbe6b7f 3014@noindent
ff7571c0
JD
3015Bison will insert the @code{trace_token} prototype into both the
3016parser header file and the parser implementation file after the
3017definitions for @code{yytokentype}, @code{YYLTYPE}, and
3018@code{YYSTYPE}.
2cbe6b7f 3019
ff7571c0
JD
3020The above examples are careful to write directives in an order that
3021reflects the layout of the generated parser implementation and header
3022files: @code{%code top}, @code{%code requires}, @code{%code provides},
3023and then @code{%code}. While your grammar files may generally be
3024easier to read if you also follow this order, Bison does not require
3025it. Instead, Bison lets you choose an organization that makes sense
3026to you.
2cbe6b7f 3027
a501eca9 3028You may declare any of these directives multiple times in the grammar file.
2cbe6b7f
JD
3029In that case, Bison concatenates the contained code in declaration order.
3030This is the only way in which the position of one of these directives within
3031the grammar file affects its functionality.
3032
3033The result of the previous two properties is greater flexibility in how you may
3034organize your grammar file.
3035For example, you may organize semantic-type-related directives by semantic
3036type:
3037
3038@smallexample
16dc6a9e 3039%code requires @{ #include "type1.h" @}
2cbe6b7f
JD
3040%union @{ type1 field1; @}
3041%destructor @{ type1_free ($$); @} <field1>
3042%printer @{ type1_print ($$); @} <field1>
3043
16dc6a9e 3044%code requires @{ #include "type2.h" @}
2cbe6b7f
JD
3045%union @{ type2 field2; @}
3046%destructor @{ type2_free ($$); @} <field2>
3047%printer @{ type2_print ($$); @} <field2>
3048@end smallexample
3049
3050@noindent
3051You could even place each of the above directive groups in the rules section of
3052the grammar file next to the set of rules that uses the associated semantic
3053type.
61fee93e
JD
3054(In the rules section, you must terminate each of those directives with a
3055semicolon.)
2cbe6b7f
JD
3056And you don't have to worry that some directive (like a @code{%union}) in the
3057definitions section is going to adversely affect their functionality in some
3058counter-intuitive manner just because it comes first.
3059Such an organization is not possible using @var{Prologue} sections.
3060
a501eca9 3061This section has been concerned with explaining the advantages of the four
8e0a5e9e 3062@var{Prologue} alternatives over the original Yacc @var{Prologue}.
a501eca9
JD
3063However, in most cases when using these directives, you shouldn't need to
3064think about all the low-level ordering issues discussed here.
3065Instead, you should simply use these directives to label each block of your
3066code according to its purpose and let Bison handle the ordering.
3067@code{%code} is the most generic label.
16dc6a9e
JD
3068Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
3069as needed.
a501eca9 3070
342b8b6e 3071@node Bison Declarations
bfa74976
RS
3072@subsection The Bison Declarations Section
3073@cindex Bison declarations (introduction)
3074@cindex declarations, Bison (introduction)
3075
3076The @var{Bison declarations} section contains declarations that define
3077terminal and nonterminal symbols, specify precedence, and so on.
3078In some simple grammars you may not need any declarations.
3079@xref{Declarations, ,Bison Declarations}.
3080
342b8b6e 3081@node Grammar Rules
bfa74976
RS
3082@subsection The Grammar Rules Section
3083@cindex grammar rules section
3084@cindex rules section for grammar
3085
3086The @dfn{grammar rules} section contains one or more Bison grammar
3087rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
3088
3089There must always be at least one grammar rule, and the first
3090@samp{%%} (which precedes the grammar rules) may never be omitted even
3091if it is the first thing in the file.
3092
38a92d50 3093@node Epilogue
75f5aaea 3094@subsection The epilogue
bfa74976 3095@cindex additional C code section
75f5aaea 3096@cindex epilogue
bfa74976
RS
3097@cindex C code, section for additional
3098
ff7571c0
JD
3099The @var{Epilogue} is copied verbatim to the end of the parser
3100implementation file, just as the @var{Prologue} is copied to the
3101beginning. This is the most convenient place to put anything that you
3102want to have in the parser implementation file but which need not come
3103before the definition of @code{yyparse}. For example, the definitions
3104of @code{yylex} and @code{yyerror} often go here. Because C requires
3105functions to be declared before being used, you often need to declare
3106functions like @code{yylex} and @code{yyerror} in the Prologue, even
3107if you define them in the Epilogue. @xref{Interface, ,Parser
3108C-Language Interface}.
bfa74976
RS
3109
3110If the last section is empty, you may omit the @samp{%%} that separates it
3111from the grammar rules.
3112
f8e1c9e5
AD
3113The Bison parser itself contains many macros and identifiers whose names
3114start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3115any such names (except those documented in this manual) in the epilogue
3116of the grammar file.
bfa74976 3117
342b8b6e 3118@node Symbols
bfa74976
RS
3119@section Symbols, Terminal and Nonterminal
3120@cindex nonterminal symbol
3121@cindex terminal symbol
3122@cindex token type
3123@cindex symbol
3124
3125@dfn{Symbols} in Bison grammars represent the grammatical classifications
3126of the language.
3127
3128A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3129class of syntactically equivalent tokens. You use the symbol in grammar
3130rules to mean that a token in that class is allowed. The symbol is
3131represented in the Bison parser by a numeric code, and the @code{yylex}
f8e1c9e5
AD
3132function returns a token type code to indicate what kind of token has
3133been read. You don't need to know what the code value is; you can use
3134the symbol to stand for it.
bfa74976 3135
f8e1c9e5
AD
3136A @dfn{nonterminal symbol} stands for a class of syntactically
3137equivalent groupings. The symbol name is used in writing grammar rules.
3138By convention, it should be all lower case.
bfa74976 3139
82f3355e
JD
3140Symbol names can contain letters, underscores, periods, and non-initial
3141digits and dashes. Dashes in symbol names are a GNU extension, incompatible
3142with POSIX Yacc. Periods and dashes make symbol names less convenient to
3143use with named references, which require brackets around such names
3144(@pxref{Named References}). Terminal symbols that contain periods or dashes
3145make little sense: since they are not valid symbols (in most programming
3146languages) they are not exported as token names.
bfa74976 3147
931c7513 3148There are three ways of writing terminal symbols in the grammar:
bfa74976
RS
3149
3150@itemize @bullet
3151@item
3152A @dfn{named token type} is written with an identifier, like an
c827f760 3153identifier in C@. By convention, it should be all upper case. Each
bfa74976
RS
3154such name must be defined with a Bison declaration such as
3155@code{%token}. @xref{Token Decl, ,Token Type Names}.
3156
3157@item
3158@cindex character token
3159@cindex literal token
3160@cindex single-character literal
931c7513
RS
3161A @dfn{character token type} (or @dfn{literal character token}) is
3162written in the grammar using the same syntax used in C for character
3163constants; for example, @code{'+'} is a character token type. A
3164character token type doesn't need to be declared unless you need to
3165specify its semantic value data type (@pxref{Value Type, ,Data Types of
3166Semantic Values}), associativity, or precedence (@pxref{Precedence,
3167,Operator Precedence}).
bfa74976
RS
3168
3169By convention, a character token type is used only to represent a
3170token that consists of that particular character. Thus, the token
3171type @code{'+'} is used to represent the character @samp{+} as a
3172token. Nothing enforces this convention, but if you depart from it,
3173your program will confuse other readers.
3174
3175All the usual escape sequences used in character literals in C can be
3176used in Bison as well, but you must not use the null character as a
72d2299c
PE
3177character literal because its numeric code, zero, signifies
3178end-of-input (@pxref{Calling Convention, ,Calling Convention
2bfc2e2a
PE
3179for @code{yylex}}). Also, unlike standard C, trigraphs have no
3180special meaning in Bison character literals, nor is backslash-newline
3181allowed.
931c7513
RS
3182
3183@item
3184@cindex string token
3185@cindex literal string token
9ecbd125 3186@cindex multicharacter literal
931c7513
RS
3187A @dfn{literal string token} is written like a C string constant; for
3188example, @code{"<="} is a literal string token. A literal string token
3189doesn't need to be declared unless you need to specify its semantic
14ded682 3190value data type (@pxref{Value Type}), associativity, or precedence
931c7513
RS
3191(@pxref{Precedence}).
3192
3193You can associate the literal string token with a symbolic name as an
3194alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3195Declarations}). If you don't do that, the lexical analyzer has to
3196retrieve the token number for the literal string token from the
3197@code{yytname} table (@pxref{Calling Convention}).
3198
c827f760 3199@strong{Warning}: literal string tokens do not work in Yacc.
931c7513
RS
3200
3201By convention, a literal string token is used only to represent a token
3202that consists of that particular string. Thus, you should use the token
3203type @code{"<="} to represent the string @samp{<=} as a token. Bison
9ecbd125 3204does not enforce this convention, but if you depart from it, people who
931c7513
RS
3205read your program will be confused.
3206
3207All the escape sequences used in string literals in C can be used in
92ac3705
PE
3208Bison as well, except that you must not use a null character within a
3209string literal. Also, unlike Standard C, trigraphs have no special
2bfc2e2a
PE
3210meaning in Bison string literals, nor is backslash-newline allowed. A
3211literal string token must contain two or more characters; for a token
3212containing just one character, use a character token (see above).
bfa74976
RS
3213@end itemize
3214
3215How you choose to write a terminal symbol has no effect on its
3216grammatical meaning. That depends only on where it appears in rules and
3217on when the parser function returns that symbol.
3218
72d2299c
PE
3219The value returned by @code{yylex} is always one of the terminal
3220symbols, except that a zero or negative value signifies end-of-input.
3221Whichever way you write the token type in the grammar rules, you write
3222it the same way in the definition of @code{yylex}. The numeric code
3223for a character token type is simply the positive numeric code of the
3224character, so @code{yylex} can use the identical value to generate the
3225requisite code, though you may need to convert it to @code{unsigned
3226char} to avoid sign-extension on hosts where @code{char} is signed.
ff7571c0
JD
3227Each named token type becomes a C macro in the parser implementation
3228file, so @code{yylex} can use the name to stand for the code. (This
3229is why periods don't make sense in terminal symbols.) @xref{Calling
3230Convention, ,Calling Convention for @code{yylex}}.
bfa74976
RS
3231
3232If @code{yylex} is defined in a separate file, you need to arrange for the
3233token-type macro definitions to be available there. Use the @samp{-d}
3234option when you run Bison, so that it will write these macro definitions
3235into a separate header file @file{@var{name}.tab.h} which you can include
3236in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3237
72d2299c 3238If you want to write a grammar that is portable to any Standard C
9d9b8b70 3239host, you must use only nonnull character tokens taken from the basic
c827f760 3240execution character set of Standard C@. This set consists of the ten
72d2299c
PE
3241digits, the 52 lower- and upper-case English letters, and the
3242characters in the following C-language string:
3243
3244@example
3245"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3246@end example
3247
f8e1c9e5
AD
3248The @code{yylex} function and Bison must use a consistent character set
3249and encoding for character tokens. For example, if you run Bison in an
8a4281b9 3250ASCII environment, but then compile and run the resulting
f8e1c9e5 3251program in an environment that uses an incompatible character set like
8a4281b9
JD
3252EBCDIC, the resulting program may not work because the tables
3253generated by Bison will assume ASCII numeric values for
f8e1c9e5
AD
3254character tokens. It is standard practice for software distributions to
3255contain C source files that were generated by Bison in an
8a4281b9
JD
3256ASCII environment, so installers on platforms that are
3257incompatible with ASCII must rebuild those files before
f8e1c9e5 3258compiling them.
e966383b 3259
bfa74976
RS
3260The symbol @code{error} is a terminal symbol reserved for error recovery
3261(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
23c5a174
AD
3262In particular, @code{yylex} should never return this value. The default
3263value of the error token is 256, unless you explicitly assigned 256 to
3264one of your tokens with a @code{%token} declaration.
bfa74976 3265
342b8b6e 3266@node Rules
bfa74976
RS
3267@section Syntax of Grammar Rules
3268@cindex rule syntax
3269@cindex grammar rule syntax
3270@cindex syntax of grammar rules
3271
3272A Bison grammar rule has the following general form:
3273
3274@example
e425e872 3275@group
bfa74976
RS
3276@var{result}: @var{components}@dots{}
3277 ;
e425e872 3278@end group
bfa74976
RS
3279@end example
3280
3281@noindent
9ecbd125 3282where @var{result} is the nonterminal symbol that this rule describes,
bfa74976 3283and @var{components} are various terminal and nonterminal symbols that
13863333 3284are put together by this rule (@pxref{Symbols}).
bfa74976
RS
3285
3286For example,
3287
3288@example
3289@group
3290exp: exp '+' exp
3291 ;
3292@end group
3293@end example
3294
3295@noindent
3296says that two groupings of type @code{exp}, with a @samp{+} token in between,
3297can be combined into a larger grouping of type @code{exp}.
3298
72d2299c
PE
3299White space in rules is significant only to separate symbols. You can add
3300extra white space as you wish.
bfa74976
RS
3301
3302Scattered among the components can be @var{actions} that determine
3303the semantics of the rule. An action looks like this:
3304
3305@example
3306@{@var{C statements}@}
3307@end example
3308
3309@noindent
287c78f6
PE
3310@cindex braced code
3311This is an example of @dfn{braced code}, that is, C code surrounded by
3312braces, much like a compound statement in C@. Braced code can contain
3313any sequence of C tokens, so long as its braces are balanced. Bison
3314does not check the braced code for correctness directly; it merely
ff7571c0
JD
3315copies the code to the parser implementation file, where the C
3316compiler can check it.
287c78f6
PE
3317
3318Within braced code, the balanced-brace count is not affected by braces
3319within comments, string literals, or character constants, but it is
3320affected by the C digraphs @samp{<%} and @samp{%>} that represent
3321braces. At the top level braced code must be terminated by @samp{@}}
3322and not by a digraph. Bison does not look for trigraphs, so if braced
3323code uses trigraphs you should ensure that they do not affect the
3324nesting of braces or the boundaries of comments, string literals, or
3325character constants.
3326
bfa74976
RS
3327Usually there is only one action and it follows the components.
3328@xref{Actions}.
3329
3330@findex |
3331Multiple rules for the same @var{result} can be written separately or can
3332be joined with the vertical-bar character @samp{|} as follows:
3333
bfa74976
RS
3334@example
3335@group
3336@var{result}: @var{rule1-components}@dots{}
3337 | @var{rule2-components}@dots{}
3338 @dots{}
3339 ;
3340@end group
3341@end example
bfa74976
RS
3342
3343@noindent
3344They are still considered distinct rules even when joined in this way.
3345
3346If @var{components} in a rule is empty, it means that @var{result} can
3347match the empty string. For example, here is how to define a
3348comma-separated sequence of zero or more @code{exp} groupings:
3349
3350@example
3351@group
3352expseq: /* empty */
3353 | expseq1
3354 ;
3355@end group
3356
3357@group
3358expseq1: exp
3359 | expseq1 ',' exp
3360 ;
3361@end group
3362@end example
3363
3364@noindent
3365It is customary to write a comment @samp{/* empty */} in each rule
3366with no components.
3367
342b8b6e 3368@node Recursion
bfa74976
RS
3369@section Recursive Rules
3370@cindex recursive rule
3371
f8e1c9e5
AD
3372A rule is called @dfn{recursive} when its @var{result} nonterminal
3373appears also on its right hand side. Nearly all Bison grammars need to
3374use recursion, because that is the only way to define a sequence of any
3375number of a particular thing. Consider this recursive definition of a
9ecbd125 3376comma-separated sequence of one or more expressions:
bfa74976
RS
3377
3378@example
3379@group
3380expseq1: exp
3381 | expseq1 ',' exp
3382 ;
3383@end group
3384@end example
3385
3386@cindex left recursion
3387@cindex right recursion
3388@noindent
3389Since the recursive use of @code{expseq1} is the leftmost symbol in the
3390right hand side, we call this @dfn{left recursion}. By contrast, here
3391the same construct is defined using @dfn{right recursion}:
3392
3393@example
3394@group
3395expseq1: exp
3396 | exp ',' expseq1
3397 ;
3398@end group
3399@end example
3400
3401@noindent
ec3bc396
AD
3402Any kind of sequence can be defined using either left recursion or right
3403recursion, but you should always use left recursion, because it can
3404parse a sequence of any number of elements with bounded stack space.
3405Right recursion uses up space on the Bison stack in proportion to the
3406number of elements in the sequence, because all the elements must be
3407shifted onto the stack before the rule can be applied even once.
3408@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3409of this.
bfa74976
RS
3410
3411@cindex mutual recursion
3412@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3413rule does not appear directly on its right hand side, but does appear
3414in rules for other nonterminals which do appear on its right hand
13863333 3415side.
bfa74976
RS
3416
3417For example:
3418
3419@example
3420@group
3421expr: primary
3422 | primary '+' primary
3423 ;
3424@end group
3425
3426@group
3427primary: constant
3428 | '(' expr ')'
3429 ;
3430@end group
3431@end example
3432
3433@noindent
3434defines two mutually-recursive nonterminals, since each refers to the
3435other.
3436
342b8b6e 3437@node Semantics
bfa74976
RS
3438@section Defining Language Semantics
3439@cindex defining language semantics
13863333 3440@cindex language semantics, defining
bfa74976
RS
3441
3442The grammar rules for a language determine only the syntax. The semantics
3443are determined by the semantic values associated with various tokens and
3444groupings, and by the actions taken when various groupings are recognized.
3445
3446For example, the calculator calculates properly because the value
3447associated with each expression is the proper number; it adds properly
3448because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3449the numbers associated with @var{x} and @var{y}.
3450
3451@menu
3452* Value Type:: Specifying one data type for all semantic values.
3453* Multiple Types:: Specifying several alternative data types.
3454* Actions:: An action is the semantic definition of a grammar rule.
3455* Action Types:: Specifying data types for actions to operate on.
3456* Mid-Rule Actions:: Most actions go at the end of a rule.
3457 This says when, why and how to use the exceptional
3458 action in the middle of a rule.
d013372c 3459* Named References:: Using named references in actions.
bfa74976
RS
3460@end menu
3461
342b8b6e 3462@node Value Type
bfa74976
RS
3463@subsection Data Types of Semantic Values
3464@cindex semantic value type
3465@cindex value type, semantic
3466@cindex data types of semantic values
3467@cindex default data type
3468
3469In a simple program it may be sufficient to use the same data type for
3470the semantic values of all language constructs. This was true in the
8a4281b9 3471RPN and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
1964ad8c 3472Notation Calculator}).
bfa74976 3473
ddc8ede1
PE
3474Bison normally uses the type @code{int} for semantic values if your
3475program uses the same data type for all language constructs. To
bfa74976
RS
3476specify some other type, define @code{YYSTYPE} as a macro, like this:
3477
3478@example
3479#define YYSTYPE double
3480@end example
3481
3482@noindent
50cce58e
PE
3483@code{YYSTYPE}'s replacement list should be a type name
3484that does not contain parentheses or square brackets.
342b8b6e 3485This macro definition must go in the prologue of the grammar file
75f5aaea 3486(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
bfa74976 3487
342b8b6e 3488@node Multiple Types
bfa74976
RS
3489@subsection More Than One Value Type
3490
3491In most programs, you will need different data types for different kinds
3492of tokens and groupings. For example, a numeric constant may need type
f8e1c9e5
AD
3493@code{int} or @code{long int}, while a string constant needs type
3494@code{char *}, and an identifier might need a pointer to an entry in the
3495symbol table.
bfa74976
RS
3496
3497To use more than one data type for semantic values in one parser, Bison
3498requires you to do two things:
3499
3500@itemize @bullet
3501@item
ddc8ede1 3502Specify the entire collection of possible data types, either by using the
704a47c4 3503@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
ddc8ede1
PE
3504Value Types}), or by using a @code{typedef} or a @code{#define} to
3505define @code{YYSTYPE} to be a union type whose member names are
3506the type tags.
bfa74976
RS
3507
3508@item
14ded682
AD
3509Choose one of those types for each symbol (terminal or nonterminal) for
3510which semantic values are used. This is done for tokens with the
3511@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3512and for groupings with the @code{%type} Bison declaration (@pxref{Type
3513Decl, ,Nonterminal Symbols}).
bfa74976
RS
3514@end itemize
3515
342b8b6e 3516@node Actions
bfa74976
RS
3517@subsection Actions
3518@cindex action
3519@vindex $$
3520@vindex $@var{n}
d013372c
AR
3521@vindex $@var{name}
3522@vindex $[@var{name}]
bfa74976
RS
3523
3524An action accompanies a syntactic rule and contains C code to be executed
3525each time an instance of that rule is recognized. The task of most actions
3526is to compute a semantic value for the grouping built by the rule from the
3527semantic values associated with tokens or smaller groupings.
3528
287c78f6
PE
3529An action consists of braced code containing C statements, and can be
3530placed at any position in the rule;
704a47c4
AD
3531it is executed at that position. Most rules have just one action at the
3532end of the rule, following all the components. Actions in the middle of
3533a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3534Actions, ,Actions in Mid-Rule}).
bfa74976 3535
ff7571c0
JD
3536The C code in an action can refer to the semantic values of the
3537components matched by the rule with the construct @code{$@var{n}},
3538which stands for the value of the @var{n}th component. The semantic
3539value for the grouping being constructed is @code{$$}. In addition,
3540the semantic values of symbols can be accessed with the named
3541references construct @code{$@var{name}} or @code{$[@var{name}]}.
3542Bison translates both of these constructs into expressions of the
3543appropriate type when it copies the actions into the parser
3544implementation file. @code{$$} (or @code{$@var{name}}, when it stands
3545for the current grouping) is translated to a modifiable lvalue, so it
3546can be assigned to.
bfa74976
RS
3547
3548Here is a typical example:
3549
3550@example
3551@group
3552exp: @dots{}
3553 | exp '+' exp
3554 @{ $$ = $1 + $3; @}
3555@end group
3556@end example
3557
d013372c
AR
3558Or, in terms of named references:
3559
3560@example
3561@group
3562exp[result]: @dots{}
3563 | exp[left] '+' exp[right]
3564 @{ $result = $left + $right; @}
3565@end group
3566@end example
3567
bfa74976
RS
3568@noindent
3569This rule constructs an @code{exp} from two smaller @code{exp} groupings
3570connected by a plus-sign token. In the action, @code{$1} and @code{$3}
d013372c 3571(@code{$left} and @code{$right})
bfa74976
RS
3572refer to the semantic values of the two component @code{exp} groupings,
3573which are the first and third symbols on the right hand side of the rule.
d013372c
AR
3574The sum is stored into @code{$$} (@code{$result}) so that it becomes the
3575semantic value of
bfa74976
RS
3576the addition-expression just recognized by the rule. If there were a
3577useful semantic value associated with the @samp{+} token, it could be
e0c471a9 3578referred to as @code{$2}.
bfa74976 3579
d013372c
AR
3580@xref{Named References,,Using Named References}, for more information
3581about using the named references construct.
3582
3ded9a63
AD
3583Note that the vertical-bar character @samp{|} is really a rule
3584separator, and actions are attached to a single rule. This is a
3585difference with tools like Flex, for which @samp{|} stands for either
3586``or'', or ``the same action as that of the next rule''. In the
3587following example, the action is triggered only when @samp{b} is found:
3588
3589@example
3590@group
3591a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3592@end group
3593@end example
3594
bfa74976
RS
3595@cindex default action
3596If you don't specify an action for a rule, Bison supplies a default:
72f889cc
AD
3597@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3598becomes the value of the whole rule. Of course, the default action is
3599valid only if the two data types match. There is no meaningful default
3600action for an empty rule; every empty rule must have an explicit action
3601unless the rule's value does not matter.
bfa74976
RS
3602
3603@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3604to tokens and groupings on the stack @emph{before} those that match the
3605current rule. This is a very risky practice, and to use it reliably
3606you must be certain of the context in which the rule is applied. Here
3607is a case in which you can use this reliably:
3608
3609@example
3610@group
3611foo: expr bar '+' expr @{ @dots{} @}
3612 | expr bar '-' expr @{ @dots{} @}
3613 ;
3614@end group
3615
3616@group
3617bar: /* empty */
3618 @{ previous_expr = $0; @}
3619 ;
3620@end group
3621@end example
3622
3623As long as @code{bar} is used only in the fashion shown here, @code{$0}
3624always refers to the @code{expr} which precedes @code{bar} in the
3625definition of @code{foo}.
3626
32c29292 3627@vindex yylval
742e4900 3628It is also possible to access the semantic value of the lookahead token, if
32c29292
JD
3629any, from a semantic action.
3630This semantic value is stored in @code{yylval}.
3631@xref{Action Features, ,Special Features for Use in Actions}.
3632
342b8b6e 3633@node Action Types
bfa74976
RS
3634@subsection Data Types of Values in Actions
3635@cindex action data types
3636@cindex data types in actions
3637
3638If you have chosen a single data type for semantic values, the @code{$$}
3639and @code{$@var{n}} constructs always have that data type.
3640
3641If you have used @code{%union} to specify a variety of data types, then you
3642must declare a choice among these types for each terminal or nonterminal
3643symbol that can have a semantic value. Then each time you use @code{$$} or
3644@code{$@var{n}}, its data type is determined by which symbol it refers to
e0c471a9 3645in the rule. In this example,
bfa74976
RS
3646
3647@example
3648@group
3649exp: @dots{}
3650 | exp '+' exp
3651 @{ $$ = $1 + $3; @}
3652@end group
3653@end example
3654
3655@noindent
3656@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3657have the data type declared for the nonterminal symbol @code{exp}. If
3658@code{$2} were used, it would have the data type declared for the
e0c471a9 3659terminal symbol @code{'+'}, whatever that might be.
bfa74976
RS
3660
3661Alternatively, you can specify the data type when you refer to the value,
3662by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3663reference. For example, if you have defined types as shown here:
3664
3665@example
3666@group
3667%union @{
3668 int itype;
3669 double dtype;
3670@}
3671@end group
3672@end example
3673
3674@noindent
3675then you can write @code{$<itype>1} to refer to the first subunit of the
3676rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3677
342b8b6e 3678@node Mid-Rule Actions
bfa74976
RS
3679@subsection Actions in Mid-Rule
3680@cindex actions in mid-rule
3681@cindex mid-rule actions
3682
3683Occasionally it is useful to put an action in the middle of a rule.
3684These actions are written just like usual end-of-rule actions, but they
3685are executed before the parser even recognizes the following components.
3686
3687A mid-rule action may refer to the components preceding it using
3688@code{$@var{n}}, but it may not refer to subsequent components because
3689it is run before they are parsed.
3690
3691The mid-rule action itself counts as one of the components of the rule.
3692This makes a difference when there is another action later in the same rule
3693(and usually there is another at the end): you have to count the actions
3694along with the symbols when working out which number @var{n} to use in
3695@code{$@var{n}}.
3696
3697The mid-rule action can also have a semantic value. The action can set
3698its value with an assignment to @code{$$}, and actions later in the rule
3699can refer to the value using @code{$@var{n}}. Since there is no symbol
3700to name the action, there is no way to declare a data type for the value
fdc6758b
MA
3701in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3702specify a data type each time you refer to this value.
bfa74976
RS
3703
3704There is no way to set the value of the entire rule with a mid-rule
3705action, because assignments to @code{$$} do not have that effect. The
3706only way to set the value for the entire rule is with an ordinary action
3707at the end of the rule.
3708
3709Here is an example from a hypothetical compiler, handling a @code{let}
3710statement that looks like @samp{let (@var{variable}) @var{statement}} and
3711serves to create a variable named @var{variable} temporarily for the
3712duration of @var{statement}. To parse this construct, we must put
3713@var{variable} into the symbol table while @var{statement} is parsed, then
3714remove it afterward. Here is how it is done:
3715
3716@example
3717@group
3718stmt: LET '(' var ')'
3719 @{ $<context>$ = push_context ();
3720 declare_variable ($3); @}
3721 stmt @{ $$ = $6;
3722 pop_context ($<context>5); @}
3723@end group
3724@end example
3725
3726@noindent
3727As soon as @samp{let (@var{variable})} has been recognized, the first
3728action is run. It saves a copy of the current semantic context (the
3729list of accessible variables) as its semantic value, using alternative
3730@code{context} in the data-type union. Then it calls
3731@code{declare_variable} to add the new variable to that list. Once the
3732first action is finished, the embedded statement @code{stmt} can be
3733parsed. Note that the mid-rule action is component number 5, so the
3734@samp{stmt} is component number 6.
3735
3736After the embedded statement is parsed, its semantic value becomes the
3737value of the entire @code{let}-statement. Then the semantic value from the
3738earlier action is used to restore the prior list of variables. This
3739removes the temporary @code{let}-variable from the list so that it won't
3740appear to exist while the rest of the program is parsed.
3741
841a7737
JD
3742@findex %destructor
3743@cindex discarded symbols, mid-rule actions
3744@cindex error recovery, mid-rule actions
3745In the above example, if the parser initiates error recovery (@pxref{Error
3746Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3747it might discard the previous semantic context @code{$<context>5} without
3748restoring it.
3749Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3750Discarded Symbols}).
ec5479ce
JD
3751However, Bison currently provides no means to declare a destructor specific to
3752a particular mid-rule action's semantic value.
841a7737
JD
3753
3754One solution is to bury the mid-rule action inside a nonterminal symbol and to
3755declare a destructor for that symbol:
3756
3757@example
3758@group
3759%type <context> let
3760%destructor @{ pop_context ($$); @} let
3761
3762%%
3763
3764stmt: let stmt
3765 @{ $$ = $2;
3766 pop_context ($1); @}
3767 ;
3768
3769let: LET '(' var ')'
3770 @{ $$ = push_context ();
3771 declare_variable ($3); @}
3772 ;
3773
3774@end group
3775@end example
3776
3777@noindent
3778Note that the action is now at the end of its rule.
3779Any mid-rule action can be converted to an end-of-rule action in this way, and
3780this is what Bison actually does to implement mid-rule actions.
3781
bfa74976
RS
3782Taking action before a rule is completely recognized often leads to
3783conflicts since the parser must commit to a parse in order to execute the
3784action. For example, the following two rules, without mid-rule actions,
3785can coexist in a working parser because the parser can shift the open-brace
3786token and look at what follows before deciding whether there is a
3787declaration or not:
3788
3789@example
3790@group
3791compound: '@{' declarations statements '@}'
3792 | '@{' statements '@}'
3793 ;
3794@end group
3795@end example
3796
3797@noindent
3798But when we add a mid-rule action as follows, the rules become nonfunctional:
3799
3800@example
3801@group
3802compound: @{ prepare_for_local_variables (); @}
3803 '@{' declarations statements '@}'
3804@end group
3805@group
3806 | '@{' statements '@}'
3807 ;
3808@end group
3809@end example
3810
3811@noindent
3812Now the parser is forced to decide whether to run the mid-rule action
3813when it has read no farther than the open-brace. In other words, it
3814must commit to using one rule or the other, without sufficient
3815information to do it correctly. (The open-brace token is what is called
742e4900
JD
3816the @dfn{lookahead} token at this time, since the parser is still
3817deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
bfa74976
RS
3818
3819You might think that you could correct the problem by putting identical
3820actions into the two rules, like this:
3821
3822@example
3823@group
3824compound: @{ prepare_for_local_variables (); @}
3825 '@{' declarations statements '@}'
3826 | @{ prepare_for_local_variables (); @}
3827 '@{' statements '@}'
3828 ;
3829@end group
3830@end example
3831
3832@noindent
3833But this does not help, because Bison does not realize that the two actions
3834are identical. (Bison never tries to understand the C code in an action.)
3835
3836If the grammar is such that a declaration can be distinguished from a
3837statement by the first token (which is true in C), then one solution which
3838does work is to put the action after the open-brace, like this:
3839
3840@example
3841@group
3842compound: '@{' @{ prepare_for_local_variables (); @}
3843 declarations statements '@}'
3844 | '@{' statements '@}'
3845 ;
3846@end group
3847@end example
3848
3849@noindent
3850Now the first token of the following declaration or statement,
3851which would in any case tell Bison which rule to use, can still do so.
3852
3853Another solution is to bury the action inside a nonterminal symbol which
3854serves as a subroutine:
3855
3856@example
3857@group
3858subroutine: /* empty */
3859 @{ prepare_for_local_variables (); @}
3860 ;
3861
3862@end group
3863
3864@group
3865compound: subroutine
3866 '@{' declarations statements '@}'
3867 | subroutine
3868 '@{' statements '@}'
3869 ;
3870@end group
3871@end example
3872
3873@noindent
3874Now Bison can execute the action in the rule for @code{subroutine} without
841a7737 3875deciding which rule for @code{compound} it will eventually use.
bfa74976 3876
d013372c
AR
3877@node Named References
3878@subsection Using Named References
3879@cindex named references
3880
3881While every semantic value can be accessed with positional references
3882@code{$@var{n}} and @code{$$}, it's often much more convenient to refer to
3883them by name. First of all, original symbol names may be used as named
3884references. For example:
3885
3886@example
3887@group
3888invocation: op '(' args ')'
3889 @{ $invocation = new_invocation ($op, $args, @@invocation); @}
3890@end group
3891@end example
3892
3893@noindent
3894The positional @code{$$}, @code{@@$}, @code{$n}, and @code{@@n} can be
3895mixed with @code{$name} and @code{@@name} arbitrarily. For example:
3896
3897@example
3898@group
3899invocation: op '(' args ')'
3900 @{ $$ = new_invocation ($op, $args, @@$); @}
3901@end group
3902@end example
3903
3904@noindent
3905However, sometimes regular symbol names are not sufficient due to
3906ambiguities:
3907
3908@example
3909@group
3910exp: exp '/' exp
3911 @{ $exp = $exp / $exp; @} // $exp is ambiguous.
3912
3913exp: exp '/' exp
3914 @{ $$ = $1 / $exp; @} // One usage is ambiguous.
3915
3916exp: exp '/' exp
3917 @{ $$ = $1 / $3; @} // No error.
3918@end group
3919@end example
3920
3921@noindent
3922When ambiguity occurs, explicitly declared names may be used for values and
3923locations. Explicit names are declared as a bracketed name after a symbol
3924appearance in rule definitions. For example:
3925@example
3926@group
3927exp[result]: exp[left] '/' exp[right]
3928 @{ $result = $left / $right; @}
3929@end group
3930@end example
3931
3932@noindent
3933Explicit names may be declared for RHS and for LHS symbols as well. In order
3934to access a semantic value generated by a mid-rule action, an explicit name
3935may also be declared by putting a bracketed name after the closing brace of
3936the mid-rule action code:
3937@example
3938@group
3939exp[res]: exp[x] '+' @{$left = $x;@}[left] exp[right]
3940 @{ $res = $left + $right; @}
3941@end group
3942@end example
3943
3944@noindent
3945
3946In references, in order to specify names containing dots and dashes, an explicit
3947bracketed syntax @code{$[name]} and @code{@@[name]} must be used:
3948@example
3949@group
3950if-stmt: IF '(' expr ')' THEN then.stmt ';'
3951 @{ $[if-stmt] = new_if_stmt ($expr, $[then.stmt]); @}
3952@end group
3953@end example
3954
3955It often happens that named references are followed by a dot, dash or other
3956C punctuation marks and operators. By default, Bison will read
3957@code{$name.suffix} as a reference to symbol value @code{$name} followed by
3958@samp{.suffix}, i.e., an access to the @samp{suffix} field of the semantic
3959value. In order to force Bison to recognize @code{name.suffix} in its entirety
3960as the name of a semantic value, bracketed syntax @code{$[name.suffix]}
3961must be used.
3962
3963
342b8b6e 3964@node Locations
847bf1f5
AD
3965@section Tracking Locations
3966@cindex location
95923bd6
AD
3967@cindex textual location
3968@cindex location, textual
847bf1f5
AD
3969
3970Though grammar rules and semantic actions are enough to write a fully
72d2299c 3971functional parser, it can be useful to process some additional information,
3e259915
MA
3972especially symbol locations.
3973
704a47c4
AD
3974The way locations are handled is defined by providing a data type, and
3975actions to take when rules are matched.
847bf1f5
AD
3976
3977@menu
3978* Location Type:: Specifying a data type for locations.
3979* Actions and Locations:: Using locations in actions.
3980* Location Default Action:: Defining a general way to compute locations.
3981@end menu
3982
342b8b6e 3983@node Location Type
847bf1f5
AD
3984@subsection Data Type of Locations
3985@cindex data type of locations
3986@cindex default location type
3987
3988Defining a data type for locations is much simpler than for semantic values,
3989since all tokens and groupings always use the same type.
3990
50cce58e
PE
3991You can specify the type of locations by defining a macro called
3992@code{YYLTYPE}, just as you can specify the semantic value type by
ddc8ede1 3993defining a @code{YYSTYPE} macro (@pxref{Value Type}).
847bf1f5
AD
3994When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3995four members:
3996
3997@example
6273355b 3998typedef struct YYLTYPE
847bf1f5
AD
3999@{
4000 int first_line;
4001 int first_column;
4002 int last_line;
4003 int last_column;
6273355b 4004@} YYLTYPE;
847bf1f5
AD
4005@end example
4006
d59e456d
AD
4007When @code{YYLTYPE} is not defined, at the beginning of the parsing, Bison
4008initializes all these fields to 1 for @code{yylloc}. To initialize
4009@code{yylloc} with a custom location type (or to chose a different
4010initialization), use the @code{%initial-action} directive. @xref{Initial
4011Action Decl, , Performing Actions before Parsing}.
cd48d21d 4012
342b8b6e 4013@node Actions and Locations
847bf1f5
AD
4014@subsection Actions and Locations
4015@cindex location actions
4016@cindex actions, location
4017@vindex @@$
4018@vindex @@@var{n}
d013372c
AR
4019@vindex @@@var{name}
4020@vindex @@[@var{name}]
847bf1f5
AD
4021
4022Actions are not only useful for defining language semantics, but also for
4023describing the behavior of the output parser with locations.
4024
4025The most obvious way for building locations of syntactic groupings is very
72d2299c 4026similar to the way semantic values are computed. In a given rule, several
847bf1f5
AD
4027constructs can be used to access the locations of the elements being matched.
4028The location of the @var{n}th component of the right hand side is
4029@code{@@@var{n}}, while the location of the left hand side grouping is
4030@code{@@$}.
4031
d013372c
AR
4032In addition, the named references construct @code{@@@var{name}} and
4033@code{@@[@var{name}]} may also be used to address the symbol locations.
4034@xref{Named References,,Using Named References}, for more information
4035about using the named references construct.
4036
3e259915 4037Here is a basic example using the default data type for locations:
847bf1f5
AD
4038
4039@example
4040@group
4041exp: @dots{}
3e259915 4042 | exp '/' exp
847bf1f5 4043 @{
3e259915
MA
4044 @@$.first_column = @@1.first_column;
4045 @@$.first_line = @@1.first_line;
847bf1f5
AD
4046 @@$.last_column = @@3.last_column;
4047 @@$.last_line = @@3.last_line;
3e259915
MA
4048 if ($3)
4049 $$ = $1 / $3;
4050 else
4051 @{
4052 $$ = 1;
4e03e201
AD
4053 fprintf (stderr,
4054 "Division by zero, l%d,c%d-l%d,c%d",
4055 @@3.first_line, @@3.first_column,
4056 @@3.last_line, @@3.last_column);
3e259915 4057 @}
847bf1f5
AD
4058 @}
4059@end group
4060@end example
4061
3e259915 4062As for semantic values, there is a default action for locations that is
72d2299c 4063run each time a rule is matched. It sets the beginning of @code{@@$} to the
3e259915 4064beginning of the first symbol, and the end of @code{@@$} to the end of the
79282c6c 4065last symbol.
3e259915 4066
72d2299c 4067With this default action, the location tracking can be fully automatic. The
3e259915
MA
4068example above simply rewrites this way:
4069
4070@example
4071@group
4072exp: @dots{}
4073 | exp '/' exp
4074 @{
4075 if ($3)
4076 $$ = $1 / $3;
4077 else
4078 @{
4079 $$ = 1;
4e03e201
AD
4080 fprintf (stderr,
4081 "Division by zero, l%d,c%d-l%d,c%d",
4082 @@3.first_line, @@3.first_column,
4083 @@3.last_line, @@3.last_column);
3e259915
MA
4084 @}
4085 @}
4086@end group
4087@end example
847bf1f5 4088
32c29292 4089@vindex yylloc
742e4900 4090It is also possible to access the location of the lookahead token, if any,
32c29292
JD
4091from a semantic action.
4092This location is stored in @code{yylloc}.
4093@xref{Action Features, ,Special Features for Use in Actions}.
4094
342b8b6e 4095@node Location Default Action
847bf1f5
AD
4096@subsection Default Action for Locations
4097@vindex YYLLOC_DEFAULT
8a4281b9 4098@cindex GLR parsers and @code{YYLLOC_DEFAULT}
847bf1f5 4099
72d2299c 4100Actually, actions are not the best place to compute locations. Since
704a47c4
AD
4101locations are much more general than semantic values, there is room in
4102the output parser to redefine the default action to take for each
72d2299c 4103rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
96b93a3d
PE
4104matched, before the associated action is run. It is also invoked
4105while processing a syntax error, to compute the error's location.
8a4281b9 4106Before reporting an unresolvable syntactic ambiguity, a GLR
8710fc41
JD
4107parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
4108of that ambiguity.
847bf1f5 4109
3e259915 4110Most of the time, this macro is general enough to suppress location
79282c6c 4111dedicated code from semantic actions.
847bf1f5 4112
72d2299c 4113The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
96b93a3d 4114the location of the grouping (the result of the computation). When a
766de5eb 4115rule is matched, the second parameter identifies locations of
96b93a3d 4116all right hand side elements of the rule being matched, and the third
8710fc41 4117parameter is the size of the rule's right hand side.
8a4281b9 4118When a GLR parser reports an ambiguity, which of multiple candidate
8710fc41
JD
4119right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
4120When processing a syntax error, the second parameter identifies locations
4121of the symbols that were discarded during error processing, and the third
96b93a3d 4122parameter is the number of discarded symbols.
847bf1f5 4123
766de5eb 4124By default, @code{YYLLOC_DEFAULT} is defined this way:
847bf1f5 4125
766de5eb 4126@smallexample
847bf1f5 4127@group
766de5eb
PE
4128# define YYLLOC_DEFAULT(Current, Rhs, N) \
4129 do \
4130 if (N) \
4131 @{ \
4132 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
4133 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
4134 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
4135 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
4136 @} \
4137 else \
4138 @{ \
4139 (Current).first_line = (Current).last_line = \
4140 YYRHSLOC(Rhs, 0).last_line; \
4141 (Current).first_column = (Current).last_column = \
4142 YYRHSLOC(Rhs, 0).last_column; \
4143 @} \
4144 while (0)
847bf1f5 4145@end group
766de5eb 4146@end smallexample
676385e2 4147
766de5eb
PE
4148where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
4149in @var{rhs} when @var{k} is positive, and the location of the symbol
f28ac696 4150just before the reduction when @var{k} and @var{n} are both zero.
676385e2 4151
3e259915 4152When defining @code{YYLLOC_DEFAULT}, you should consider that:
847bf1f5 4153
3e259915 4154@itemize @bullet
79282c6c 4155@item
72d2299c 4156All arguments are free of side-effects. However, only the first one (the
3e259915 4157result) should be modified by @code{YYLLOC_DEFAULT}.
847bf1f5 4158
3e259915 4159@item
766de5eb
PE
4160For consistency with semantic actions, valid indexes within the
4161right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
4162valid index, and it refers to the symbol just before the reduction.
4163During error processing @var{n} is always positive.
0ae99356
PE
4164
4165@item
4166Your macro should parenthesize its arguments, if need be, since the
4167actual arguments may not be surrounded by parentheses. Also, your
4168macro should expand to something that can be used as a single
4169statement when it is followed by a semicolon.
3e259915 4170@end itemize
847bf1f5 4171
342b8b6e 4172@node Declarations
bfa74976
RS
4173@section Bison Declarations
4174@cindex declarations, Bison
4175@cindex Bison declarations
4176
4177The @dfn{Bison declarations} section of a Bison grammar defines the symbols
4178used in formulating the grammar and the data types of semantic values.
4179@xref{Symbols}.
4180
4181All token type names (but not single-character literal tokens such as
4182@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
4183declared if you need to specify which data type to use for the semantic
4184value (@pxref{Multiple Types, ,More Than One Value Type}).
4185
ff7571c0
JD
4186The first rule in the grammar file also specifies the start symbol, by
4187default. If you want some other symbol to be the start symbol, you
4188must declare it explicitly (@pxref{Language and Grammar, ,Languages
4189and Context-Free Grammars}).
bfa74976
RS
4190
4191@menu
b50d2359 4192* Require Decl:: Requiring a Bison version.
bfa74976
RS
4193* Token Decl:: Declaring terminal symbols.
4194* Precedence Decl:: Declaring terminals with precedence and associativity.
4195* Union Decl:: Declaring the set of all semantic value types.
4196* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 4197* Initial Action Decl:: Code run before parsing starts.
72f889cc 4198* Destructor Decl:: Declaring how symbols are freed.
d6328241 4199* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
4200* Start Decl:: Specifying the start symbol.
4201* Pure Decl:: Requesting a reentrant parser.
9987d1b3 4202* Push Decl:: Requesting a push parser.
bfa74976 4203* Decl Summary:: Table of all Bison declarations.
35c1e5f0 4204* %define Summary:: Defining variables to adjust Bison's behavior.
e0c07222 4205* %code Summary:: Inserting code into the parser source.
bfa74976
RS
4206@end menu
4207
b50d2359
AD
4208@node Require Decl
4209@subsection Require a Version of Bison
4210@cindex version requirement
4211@cindex requiring a version of Bison
4212@findex %require
4213
4214You may require the minimum version of Bison to process the grammar. If
9b8a5ce0
AD
4215the requirement is not met, @command{bison} exits with an error (exit
4216status 63).
b50d2359
AD
4217
4218@example
4219%require "@var{version}"
4220@end example
4221
342b8b6e 4222@node Token Decl
bfa74976
RS
4223@subsection Token Type Names
4224@cindex declaring token type names
4225@cindex token type names, declaring
931c7513 4226@cindex declaring literal string tokens
bfa74976
RS
4227@findex %token
4228
4229The basic way to declare a token type name (terminal symbol) is as follows:
4230
4231@example
4232%token @var{name}
4233@end example
4234
4235Bison will convert this into a @code{#define} directive in
4236the parser, so that the function @code{yylex} (if it is in this file)
4237can use the name @var{name} to stand for this token type's code.
4238
d78f0ac9
AD
4239Alternatively, you can use @code{%left}, @code{%right},
4240@code{%precedence}, or
14ded682
AD
4241@code{%nonassoc} instead of @code{%token}, if you wish to specify
4242associativity and precedence. @xref{Precedence Decl, ,Operator
4243Precedence}.
bfa74976
RS
4244
4245You can explicitly specify the numeric code for a token type by appending
b1cc23c4 4246a nonnegative decimal or hexadecimal integer value in the field immediately
1452af69 4247following the token name:
bfa74976
RS
4248
4249@example
4250%token NUM 300
1452af69 4251%token XNUM 0x12d // a GNU extension
bfa74976
RS
4252@end example
4253
4254@noindent
4255It is generally best, however, to let Bison choose the numeric codes for
4256all token types. Bison will automatically select codes that don't conflict
e966383b 4257with each other or with normal characters.
bfa74976
RS
4258
4259In the event that the stack type is a union, you must augment the
4260@code{%token} or other token declaration to include the data type
704a47c4
AD
4261alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4262Than One Value Type}).
bfa74976
RS
4263
4264For example:
4265
4266@example
4267@group
4268%union @{ /* define stack type */
4269 double val;
4270 symrec *tptr;
4271@}
4272%token <val> NUM /* define token NUM and its type */
4273@end group
4274@end example
4275
931c7513
RS
4276You can associate a literal string token with a token type name by
4277writing the literal string at the end of a @code{%token}
4278declaration which declares the name. For example:
4279
4280@example
4281%token arrow "=>"
4282@end example
4283
4284@noindent
4285For example, a grammar for the C language might specify these names with
4286equivalent literal string tokens:
4287
4288@example
4289%token <operator> OR "||"
4290%token <operator> LE 134 "<="
4291%left OR "<="
4292@end example
4293
4294@noindent
4295Once you equate the literal string and the token name, you can use them
4296interchangeably in further declarations or the grammar rules. The
4297@code{yylex} function can use the token name or the literal string to
4298obtain the token type code number (@pxref{Calling Convention}).
b1cc23c4
JD
4299Syntax error messages passed to @code{yyerror} from the parser will reference
4300the literal string instead of the token name.
4301
4302The token numbered as 0 corresponds to end of file; the following line
4303allows for nicer error messages referring to ``end of file'' instead
4304of ``$end'':
4305
4306@example
4307%token END 0 "end of file"
4308@end example
931c7513 4309
342b8b6e 4310@node Precedence Decl
bfa74976
RS
4311@subsection Operator Precedence
4312@cindex precedence declarations
4313@cindex declaring operator precedence
4314@cindex operator precedence, declaring
4315
d78f0ac9
AD
4316Use the @code{%left}, @code{%right}, @code{%nonassoc}, or
4317@code{%precedence} declaration to
bfa74976
RS
4318declare a token and specify its precedence and associativity, all at
4319once. These are called @dfn{precedence declarations}.
704a47c4
AD
4320@xref{Precedence, ,Operator Precedence}, for general information on
4321operator precedence.
bfa74976 4322
ab7f29f8 4323The syntax of a precedence declaration is nearly the same as that of
bfa74976
RS
4324@code{%token}: either
4325
4326@example
4327%left @var{symbols}@dots{}
4328@end example
4329
4330@noindent
4331or
4332
4333@example
4334%left <@var{type}> @var{symbols}@dots{}
4335@end example
4336
4337And indeed any of these declarations serves the purposes of @code{%token}.
4338But in addition, they specify the associativity and relative precedence for
4339all the @var{symbols}:
4340
4341@itemize @bullet
4342@item
4343The associativity of an operator @var{op} determines how repeated uses
4344of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4345@var{z}} is parsed by grouping @var{x} with @var{y} first or by
4346grouping @var{y} with @var{z} first. @code{%left} specifies
4347left-associativity (grouping @var{x} with @var{y} first) and
4348@code{%right} specifies right-associativity (grouping @var{y} with
4349@var{z} first). @code{%nonassoc} specifies no associativity, which
4350means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4351considered a syntax error.
4352
d78f0ac9
AD
4353@code{%precedence} gives only precedence to the @var{symbols}, and
4354defines no associativity at all. Use this to define precedence only,
4355and leave any potential conflict due to associativity enabled.
4356
bfa74976
RS
4357@item
4358The precedence of an operator determines how it nests with other operators.
4359All the tokens declared in a single precedence declaration have equal
4360precedence and nest together according to their associativity.
4361When two tokens declared in different precedence declarations associate,
4362the one declared later has the higher precedence and is grouped first.
4363@end itemize
4364
ab7f29f8
JD
4365For backward compatibility, there is a confusing difference between the
4366argument lists of @code{%token} and precedence declarations.
4367Only a @code{%token} can associate a literal string with a token type name.
4368A precedence declaration always interprets a literal string as a reference to a
4369separate token.
4370For example:
4371
4372@example
4373%left OR "<=" // Does not declare an alias.
4374%left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4375@end example
4376
342b8b6e 4377@node Union Decl
bfa74976
RS
4378@subsection The Collection of Value Types
4379@cindex declaring value types
4380@cindex value types, declaring
4381@findex %union
4382
287c78f6
PE
4383The @code{%union} declaration specifies the entire collection of
4384possible data types for semantic values. The keyword @code{%union} is
4385followed by braced code containing the same thing that goes inside a
4386@code{union} in C@.
bfa74976
RS
4387
4388For example:
4389
4390@example
4391@group
4392%union @{
4393 double val;
4394 symrec *tptr;
4395@}
4396@end group
4397@end example
4398
4399@noindent
4400This says that the two alternative types are @code{double} and @code{symrec
4401*}. They are given names @code{val} and @code{tptr}; these names are used
4402in the @code{%token} and @code{%type} declarations to pick one of the types
4403for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4404
8a4281b9 4405As an extension to POSIX, a tag is allowed after the
6273355b
PE
4406@code{union}. For example:
4407
4408@example
4409@group
4410%union value @{
4411 double val;
4412 symrec *tptr;
4413@}
4414@end group
4415@end example
4416
d6ca7905 4417@noindent
6273355b
PE
4418specifies the union tag @code{value}, so the corresponding C type is
4419@code{union value}. If you do not specify a tag, it defaults to
4420@code{YYSTYPE}.
4421
8a4281b9 4422As another extension to POSIX, you may specify multiple
d6ca7905
PE
4423@code{%union} declarations; their contents are concatenated. However,
4424only the first @code{%union} declaration can specify a tag.
4425
6273355b 4426Note that, unlike making a @code{union} declaration in C, you need not write
bfa74976
RS
4427a semicolon after the closing brace.
4428
ddc8ede1
PE
4429Instead of @code{%union}, you can define and use your own union type
4430@code{YYSTYPE} if your grammar contains at least one
4431@samp{<@var{type}>} tag. For example, you can put the following into
4432a header file @file{parser.h}:
4433
4434@example
4435@group
4436union YYSTYPE @{
4437 double val;
4438 symrec *tptr;
4439@};
4440typedef union YYSTYPE YYSTYPE;
4441@end group
4442@end example
4443
4444@noindent
4445and then your grammar can use the following
4446instead of @code{%union}:
4447
4448@example
4449@group
4450%@{
4451#include "parser.h"
4452%@}
4453%type <val> expr
4454%token <tptr> ID
4455@end group
4456@end example
4457
342b8b6e 4458@node Type Decl
bfa74976
RS
4459@subsection Nonterminal Symbols
4460@cindex declaring value types, nonterminals
4461@cindex value types, nonterminals, declaring
4462@findex %type
4463
4464@noindent
4465When you use @code{%union} to specify multiple value types, you must
4466declare the value type of each nonterminal symbol for which values are
4467used. This is done with a @code{%type} declaration, like this:
4468
4469@example
4470%type <@var{type}> @var{nonterminal}@dots{}
4471@end example
4472
4473@noindent
704a47c4
AD
4474Here @var{nonterminal} is the name of a nonterminal symbol, and
4475@var{type} is the name given in the @code{%union} to the alternative
4476that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4477can give any number of nonterminal symbols in the same @code{%type}
4478declaration, if they have the same value type. Use spaces to separate
4479the symbol names.
bfa74976 4480
931c7513
RS
4481You can also declare the value type of a terminal symbol. To do this,
4482use the same @code{<@var{type}>} construction in a declaration for the
4483terminal symbol. All kinds of token declarations allow
4484@code{<@var{type}>}.
4485
18d192f0
AD
4486@node Initial Action Decl
4487@subsection Performing Actions before Parsing
4488@findex %initial-action
4489
4490Sometimes your parser needs to perform some initializations before
4491parsing. The @code{%initial-action} directive allows for such arbitrary
4492code.
4493
4494@deffn {Directive} %initial-action @{ @var{code} @}
4495@findex %initial-action
287c78f6 4496Declare that the braced @var{code} must be invoked before parsing each time
451364ed 4497@code{yyparse} is called. The @var{code} may use @code{$$} and
742e4900 4498@code{@@$} --- initial value and location of the lookahead --- and the
451364ed 4499@code{%parse-param}.
18d192f0
AD
4500@end deffn
4501
451364ed
AD
4502For instance, if your locations use a file name, you may use
4503
4504@example
48b16bbc 4505%parse-param @{ char const *file_name @};
451364ed
AD
4506%initial-action
4507@{
4626a15d 4508 @@$.initialize (file_name);
451364ed
AD
4509@};
4510@end example
4511
18d192f0 4512
72f889cc
AD
4513@node Destructor Decl
4514@subsection Freeing Discarded Symbols
4515@cindex freeing discarded symbols
4516@findex %destructor
12e35840 4517@findex <*>
3ebecc24 4518@findex <>
a85284cf
AD
4519During error recovery (@pxref{Error Recovery}), symbols already pushed
4520on the stack and tokens coming from the rest of the file are discarded
4521until the parser falls on its feet. If the parser runs out of memory,
9d9b8b70 4522or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
a85284cf
AD
4523symbols on the stack must be discarded. Even if the parser succeeds, it
4524must discard the start symbol.
258b75ca
PE
4525
4526When discarded symbols convey heap based information, this memory is
4527lost. While this behavior can be tolerable for batch parsers, such as
4b367315
AD
4528in traditional compilers, it is unacceptable for programs like shells or
4529protocol implementations that may parse and execute indefinitely.
258b75ca 4530
a85284cf
AD
4531The @code{%destructor} directive defines code that is called when a
4532symbol is automatically discarded.
72f889cc
AD
4533
4534@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4535@findex %destructor
287c78f6
PE
4536Invoke the braced @var{code} whenever the parser discards one of the
4537@var{symbols}.
4b367315 4538Within @var{code}, @code{$$} designates the semantic value associated
ec5479ce
JD
4539with the discarded symbol, and @code{@@$} designates its location.
4540The additional parser parameters are also available (@pxref{Parser Function, ,
4541The Parser Function @code{yyparse}}).
ec5479ce 4542
b2a0b7ca
JD
4543When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4544per-symbol @code{%destructor}.
4545You may also define a per-type @code{%destructor} by listing a semantic type
12e35840 4546tag among @var{symbols}.
b2a0b7ca 4547In that case, the parser will invoke this @var{code} whenever it discards any
12e35840 4548grammar symbol that has that semantic type tag unless that symbol has its own
b2a0b7ca
JD
4549per-symbol @code{%destructor}.
4550
12e35840 4551Finally, you can define two different kinds of default @code{%destructor}s.
85894313
JD
4552(These default forms are experimental.
4553More user feedback will help to determine whether they should become permanent
4554features.)
3ebecc24 4555You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
12e35840
JD
4556exactly one @code{%destructor} declaration in your grammar file.
4557The parser will invoke the @var{code} associated with one of these whenever it
4558discards any user-defined grammar symbol that has no per-symbol and no per-type
4559@code{%destructor}.
4560The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4561symbol for which you have formally declared a semantic type tag (@code{%type}
4562counts as such a declaration, but @code{$<tag>$} does not).
3ebecc24 4563The parser uses the @var{code} for @code{<>} in the case of such a grammar
12e35840 4564symbol that has no declared semantic type tag.
72f889cc
AD
4565@end deffn
4566
b2a0b7ca 4567@noindent
12e35840 4568For example:
72f889cc
AD
4569
4570@smallexample
ec5479ce
JD
4571%union @{ char *string; @}
4572%token <string> STRING1
4573%token <string> STRING2
4574%type <string> string1
4575%type <string> string2
b2a0b7ca
JD
4576%union @{ char character; @}
4577%token <character> CHR
4578%type <character> chr
12e35840
JD
4579%token TAGLESS
4580
b2a0b7ca 4581%destructor @{ @} <character>
12e35840
JD
4582%destructor @{ free ($$); @} <*>
4583%destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
3ebecc24 4584%destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
72f889cc
AD
4585@end smallexample
4586
4587@noindent
b2a0b7ca
JD
4588guarantees that, when the parser discards any user-defined symbol that has a
4589semantic type tag other than @code{<character>}, it passes its semantic value
12e35840 4590to @code{free} by default.
ec5479ce
JD
4591However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4592prints its line number to @code{stdout}.
4593It performs only the second @code{%destructor} in this case, so it invokes
4594@code{free} only once.
12e35840
JD
4595Finally, the parser merely prints a message whenever it discards any symbol,
4596such as @code{TAGLESS}, that has no semantic type tag.
4597
4598A Bison-generated parser invokes the default @code{%destructor}s only for
4599user-defined as opposed to Bison-defined symbols.
4600For example, the parser will not invoke either kind of default
4601@code{%destructor} for the special Bison-defined symbols @code{$accept},
4602@code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4603none of which you can reference in your grammar.
4604It also will not invoke either for the @code{error} token (@pxref{Table of
4605Symbols, ,error}), which is always defined by Bison regardless of whether you
4606reference it in your grammar.
4607However, it may invoke one of them for the end token (token 0) if you
4608redefine it from @code{$end} to, for example, @code{END}:
3508ce36
JD
4609
4610@smallexample
4611%token END 0
4612@end smallexample
4613
12e35840
JD
4614@cindex actions in mid-rule
4615@cindex mid-rule actions
4616Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4617mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4618That is, Bison does not consider a mid-rule to have a semantic value if you do
4619not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4620@var{n} is the RHS symbol position of the mid-rule) in any later action in that
4621rule.
4622However, if you do reference either, the Bison-generated parser will invoke the
3ebecc24 4623@code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
12e35840 4624
3508ce36
JD
4625@ignore
4626@noindent
4627In the future, it may be possible to redefine the @code{error} token as a
4628nonterminal that captures the discarded symbols.
4629In that case, the parser will invoke the default destructor for it as well.
4630@end ignore
4631
e757bb10
AD
4632@sp 1
4633
4634@cindex discarded symbols
4635@dfn{Discarded symbols} are the following:
4636
4637@itemize
4638@item
4639stacked symbols popped during the first phase of error recovery,
4640@item
4641incoming terminals during the second phase of error recovery,
4642@item
742e4900 4643the current lookahead and the entire stack (except the current
9d9b8b70 4644right-hand side symbols) when the parser returns immediately, and
258b75ca
PE
4645@item
4646the start symbol, when the parser succeeds.
e757bb10
AD
4647@end itemize
4648
9d9b8b70
PE
4649The parser can @dfn{return immediately} because of an explicit call to
4650@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4651exhaustion.
4652
29553547 4653Right-hand side symbols of a rule that explicitly triggers a syntax
9d9b8b70
PE
4654error via @code{YYERROR} are not discarded automatically. As a rule
4655of thumb, destructors are invoked only when user actions cannot manage
a85284cf 4656the memory.
e757bb10 4657
342b8b6e 4658@node Expect Decl
bfa74976
RS
4659@subsection Suppressing Conflict Warnings
4660@cindex suppressing conflict warnings
4661@cindex preventing warnings about conflicts
4662@cindex warnings, preventing
4663@cindex conflicts, suppressing warnings of
4664@findex %expect
d6328241 4665@findex %expect-rr
bfa74976
RS
4666
4667Bison normally warns if there are any conflicts in the grammar
7da99ede
AD
4668(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4669have harmless shift/reduce conflicts which are resolved in a predictable
4670way and would be difficult to eliminate. It is desirable to suppress
4671the warning about these conflicts unless the number of conflicts
4672changes. You can do this with the @code{%expect} declaration.
bfa74976
RS
4673
4674The declaration looks like this:
4675
4676@example
4677%expect @var{n}
4678@end example
4679
035aa4a0
PE
4680Here @var{n} is a decimal integer. The declaration says there should
4681be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4682Bison reports an error if the number of shift/reduce conflicts differs
4683from @var{n}, or if there are any reduce/reduce conflicts.
bfa74976 4684
eb45ef3b 4685For deterministic parsers, reduce/reduce conflicts are more
035aa4a0 4686serious, and should be eliminated entirely. Bison will always report
8a4281b9 4687reduce/reduce conflicts for these parsers. With GLR
035aa4a0 4688parsers, however, both kinds of conflicts are routine; otherwise,
8a4281b9 4689there would be no need to use GLR parsing. Therefore, it is
035aa4a0 4690also possible to specify an expected number of reduce/reduce conflicts
8a4281b9 4691in GLR parsers, using the declaration:
d6328241
PH
4692
4693@example
4694%expect-rr @var{n}
4695@end example
4696
bfa74976
RS
4697In general, using @code{%expect} involves these steps:
4698
4699@itemize @bullet
4700@item
4701Compile your grammar without @code{%expect}. Use the @samp{-v} option
4702to get a verbose list of where the conflicts occur. Bison will also
4703print the number of conflicts.
4704
4705@item
4706Check each of the conflicts to make sure that Bison's default
4707resolution is what you really want. If not, rewrite the grammar and
4708go back to the beginning.
4709
4710@item
4711Add an @code{%expect} declaration, copying the number @var{n} from the
8a4281b9 4712number which Bison printed. With GLR parsers, add an
035aa4a0 4713@code{%expect-rr} declaration as well.
bfa74976
RS
4714@end itemize
4715
93d7dde9
JD
4716Now Bison will report an error if you introduce an unexpected conflict,
4717but will keep silent otherwise.
bfa74976 4718
342b8b6e 4719@node Start Decl
bfa74976
RS
4720@subsection The Start-Symbol
4721@cindex declaring the start symbol
4722@cindex start symbol, declaring
4723@cindex default start symbol
4724@findex %start
4725
4726Bison assumes by default that the start symbol for the grammar is the first
4727nonterminal specified in the grammar specification section. The programmer
4728may override this restriction with the @code{%start} declaration as follows:
4729
4730@example
4731%start @var{symbol}
4732@end example
4733
342b8b6e 4734@node Pure Decl
bfa74976
RS
4735@subsection A Pure (Reentrant) Parser
4736@cindex reentrant parser
4737@cindex pure parser
d9df47b6 4738@findex %define api.pure
bfa74976
RS
4739
4740A @dfn{reentrant} program is one which does not alter in the course of
4741execution; in other words, it consists entirely of @dfn{pure} (read-only)
4742code. Reentrancy is important whenever asynchronous execution is possible;
9d9b8b70
PE
4743for example, a nonreentrant program may not be safe to call from a signal
4744handler. In systems with multiple threads of control, a nonreentrant
bfa74976
RS
4745program must be called only within interlocks.
4746
70811b85 4747Normally, Bison generates a parser which is not reentrant. This is
c827f760
PE
4748suitable for most uses, and it permits compatibility with Yacc. (The
4749standard Yacc interfaces are inherently nonreentrant, because they use
70811b85
RS
4750statically allocated variables for communication with @code{yylex},
4751including @code{yylval} and @code{yylloc}.)
bfa74976 4752
70811b85 4753Alternatively, you can generate a pure, reentrant parser. The Bison
67501061 4754declaration @samp{%define api.pure} says that you want the parser to be
70811b85 4755reentrant. It looks like this:
bfa74976
RS
4756
4757@example
d9df47b6 4758%define api.pure
bfa74976
RS
4759@end example
4760
70811b85
RS
4761The result is that the communication variables @code{yylval} and
4762@code{yylloc} become local variables in @code{yyparse}, and a different
4763calling convention is used for the lexical analyzer function
4764@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
f4101aa6
AD
4765Parsers}, for the details of this. The variable @code{yynerrs}
4766becomes local in @code{yyparse} in pull mode but it becomes a member
9987d1b3 4767of yypstate in push mode. (@pxref{Error Reporting, ,The Error
70811b85
RS
4768Reporting Function @code{yyerror}}). The convention for calling
4769@code{yyparse} itself is unchanged.
4770
4771Whether the parser is pure has nothing to do with the grammar rules.
4772You can generate either a pure parser or a nonreentrant parser from any
4773valid grammar.
bfa74976 4774
9987d1b3
JD
4775@node Push Decl
4776@subsection A Push Parser
4777@cindex push parser
4778@cindex push parser
67212941 4779@findex %define api.push-pull
9987d1b3 4780
59da312b
JD
4781(The current push parsing interface is experimental and may evolve.
4782More user feedback will help to stabilize it.)
4783
f4101aa6
AD
4784A pull parser is called once and it takes control until all its input
4785is completely parsed. A push parser, on the other hand, is called
9987d1b3
JD
4786each time a new token is made available.
4787
f4101aa6 4788A push parser is typically useful when the parser is part of a
9987d1b3 4789main event loop in the client's application. This is typically
f4101aa6
AD
4790a requirement of a GUI, when the main event loop needs to be triggered
4791within a certain time period.
9987d1b3 4792
d782395d
JD
4793Normally, Bison generates a pull parser.
4794The following Bison declaration says that you want the parser to be a push
35c1e5f0 4795parser (@pxref{%define Summary,,api.push-pull}):
9987d1b3
JD
4796
4797@example
cf499cff 4798%define api.push-pull push
9987d1b3
JD
4799@end example
4800
4801In almost all cases, you want to ensure that your push parser is also
4802a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
f4101aa6 4803time you should create an impure push parser is to have backwards
9987d1b3
JD
4804compatibility with the impure Yacc pull mode interface. Unless you know
4805what you are doing, your declarations should look like this:
4806
4807@example
d9df47b6 4808%define api.pure
cf499cff 4809%define api.push-pull push
9987d1b3
JD
4810@end example
4811
f4101aa6
AD
4812There is a major notable functional difference between the pure push parser
4813and the impure push parser. It is acceptable for a pure push parser to have
9987d1b3
JD
4814many parser instances, of the same type of parser, in memory at the same time.
4815An impure push parser should only use one parser at a time.
4816
4817When a push parser is selected, Bison will generate some new symbols in
f4101aa6
AD
4818the generated parser. @code{yypstate} is a structure that the generated
4819parser uses to store the parser's state. @code{yypstate_new} is the
9987d1b3
JD
4820function that will create a new parser instance. @code{yypstate_delete}
4821will free the resources associated with the corresponding parser instance.
f4101aa6 4822Finally, @code{yypush_parse} is the function that should be called whenever a
9987d1b3
JD
4823token is available to provide the parser. A trivial example
4824of using a pure push parser would look like this:
4825
4826@example
4827int status;
4828yypstate *ps = yypstate_new ();
4829do @{
4830 status = yypush_parse (ps, yylex (), NULL);
4831@} while (status == YYPUSH_MORE);
4832yypstate_delete (ps);
4833@end example
4834
4835If the user decided to use an impure push parser, a few things about
f4101aa6 4836the generated parser will change. The @code{yychar} variable becomes
9987d1b3
JD
4837a global variable instead of a variable in the @code{yypush_parse} function.
4838For this reason, the signature of the @code{yypush_parse} function is
f4101aa6 4839changed to remove the token as a parameter. A nonreentrant push parser
9987d1b3
JD
4840example would thus look like this:
4841
4842@example
4843extern int yychar;
4844int status;
4845yypstate *ps = yypstate_new ();
4846do @{
4847 yychar = yylex ();
4848 status = yypush_parse (ps);
4849@} while (status == YYPUSH_MORE);
4850yypstate_delete (ps);
4851@end example
4852
f4101aa6 4853That's it. Notice the next token is put into the global variable @code{yychar}
9987d1b3
JD
4854for use by the next invocation of the @code{yypush_parse} function.
4855
f4101aa6 4856Bison also supports both the push parser interface along with the pull parser
9987d1b3 4857interface in the same generated parser. In order to get this functionality,
cf499cff
JD
4858you should replace the @samp{%define api.push-pull push} declaration with the
4859@samp{%define api.push-pull both} declaration. Doing this will create all of
c373bf8b 4860the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
f4101aa6
AD
4861and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4862would be used. However, the user should note that it is implemented in the
d782395d
JD
4863generated parser by calling @code{yypull_parse}.
4864This makes the @code{yyparse} function that is generated with the
cf499cff 4865@samp{%define api.push-pull both} declaration slower than the normal
d782395d
JD
4866@code{yyparse} function. If the user
4867calls the @code{yypull_parse} function it will parse the rest of the input
f4101aa6
AD
4868stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4869and then @code{yypull_parse} the rest of the input stream. If you would like
4870to switch back and forth between between parsing styles, you would have to
4871write your own @code{yypull_parse} function that knows when to quit looking
4872for input. An example of using the @code{yypull_parse} function would look
9987d1b3
JD
4873like this:
4874
4875@example
4876yypstate *ps = yypstate_new ();
4877yypull_parse (ps); /* Will call the lexer */
4878yypstate_delete (ps);
4879@end example
4880
67501061 4881Adding the @samp{%define api.pure} declaration does exactly the same thing to
cf499cff
JD
4882the generated parser with @samp{%define api.push-pull both} as it did for
4883@samp{%define api.push-pull push}.
9987d1b3 4884
342b8b6e 4885@node Decl Summary
bfa74976
RS
4886@subsection Bison Declaration Summary
4887@cindex Bison declaration summary
4888@cindex declaration summary
4889@cindex summary, Bison declaration
4890
d8988b2f 4891Here is a summary of the declarations used to define a grammar:
bfa74976 4892
18b519c0 4893@deffn {Directive} %union
bfa74976
RS
4894Declare the collection of data types that semantic values may have
4895(@pxref{Union Decl, ,The Collection of Value Types}).
18b519c0 4896@end deffn
bfa74976 4897
18b519c0 4898@deffn {Directive} %token
bfa74976
RS
4899Declare a terminal symbol (token type name) with no precedence
4900or associativity specified (@pxref{Token Decl, ,Token Type Names}).
18b519c0 4901@end deffn
bfa74976 4902
18b519c0 4903@deffn {Directive} %right
bfa74976
RS
4904Declare a terminal symbol (token type name) that is right-associative
4905(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4906@end deffn
bfa74976 4907
18b519c0 4908@deffn {Directive} %left
bfa74976
RS
4909Declare a terminal symbol (token type name) that is left-associative
4910(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4911@end deffn
bfa74976 4912
18b519c0 4913@deffn {Directive} %nonassoc
bfa74976 4914Declare a terminal symbol (token type name) that is nonassociative
bfa74976 4915(@pxref{Precedence Decl, ,Operator Precedence}).
39a06c25
PE
4916Using it in a way that would be associative is a syntax error.
4917@end deffn
4918
91d2c560 4919@ifset defaultprec
39a06c25 4920@deffn {Directive} %default-prec
22fccf95 4921Assign a precedence to rules lacking an explicit @code{%prec} modifier
39a06c25
PE
4922(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4923@end deffn
91d2c560 4924@end ifset
bfa74976 4925
18b519c0 4926@deffn {Directive} %type
bfa74976
RS
4927Declare the type of semantic values for a nonterminal symbol
4928(@pxref{Type Decl, ,Nonterminal Symbols}).
18b519c0 4929@end deffn
bfa74976 4930
18b519c0 4931@deffn {Directive} %start
89cab50d
AD
4932Specify the grammar's start symbol (@pxref{Start Decl, ,The
4933Start-Symbol}).
18b519c0 4934@end deffn
bfa74976 4935
18b519c0 4936@deffn {Directive} %expect
bfa74976
RS
4937Declare the expected number of shift-reduce conflicts
4938(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
18b519c0
AD
4939@end deffn
4940
bfa74976 4941
d8988b2f
AD
4942@sp 1
4943@noindent
4944In order to change the behavior of @command{bison}, use the following
4945directives:
4946
148d66d8 4947@deffn {Directive} %code @{@var{code}@}
e0c07222 4948@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
148d66d8 4949@findex %code
e0c07222
JD
4950Insert @var{code} verbatim into the output parser source at the
4951default location or at the location specified by @var{qualifier}.
4952@xref{%code Summary}.
148d66d8
JD
4953@end deffn
4954
18b519c0 4955@deffn {Directive} %debug
fa819509
AD
4956Instrument the output parser for traces. Obsoleted by @samp{%define
4957parse.trace}.
ec3bc396 4958@xref{Tracing, ,Tracing Your Parser}.
f7dae1ea 4959@end deffn
d8988b2f 4960
35c1e5f0
JD
4961@deffn {Directive} %define @var{variable}
4962@deffnx {Directive} %define @var{variable} @var{value}
4963@deffnx {Directive} %define @var{variable} "@var{value}"
4964Define a variable to adjust Bison's behavior. @xref{%define Summary}.
4965@end deffn
4966
4967@deffn {Directive} %defines
4968Write a parser header file containing macro definitions for the token
4969type names defined in the grammar as well as a few other declarations.
4970If the parser implementation file is named @file{@var{name}.c} then
4971the parser header file is named @file{@var{name}.h}.
4972
4973For C parsers, the parser header file declares @code{YYSTYPE} unless
4974@code{YYSTYPE} is already defined as a macro or you have used a
4975@code{<@var{type}>} tag without using @code{%union}. Therefore, if
4976you are using a @code{%union} (@pxref{Multiple Types, ,More Than One
4977Value Type}) with components that require other definitions, or if you
4978have defined a @code{YYSTYPE} macro or type definition (@pxref{Value
4979Type, ,Data Types of Semantic Values}), you need to arrange for these
4980definitions to be propagated to all modules, e.g., by putting them in
4981a prerequisite header that is included both by your parser and by any
4982other module that needs @code{YYSTYPE}.
4983
4984Unless your parser is pure, the parser header file declares
4985@code{yylval} as an external variable. @xref{Pure Decl, ,A Pure
4986(Reentrant) Parser}.
4987
4988If you have also used locations, the parser header file declares
4989@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
4990the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations,
4991,Tracking Locations}.
4992
4993This parser header file is normally essential if you wish to put the
4994definition of @code{yylex} in a separate source file, because
4995@code{yylex} typically needs to be able to refer to the
4996above-mentioned declarations and to the token type codes. @xref{Token
4997Values, ,Semantic Values of Tokens}.
4998
4999@findex %code requires
5000@findex %code provides
5001If you have declared @code{%code requires} or @code{%code provides}, the output
5002header also contains their code.
5003@xref{%code Summary}.
5004@end deffn
5005
5006@deffn {Directive} %defines @var{defines-file}
5007Same as above, but save in the file @var{defines-file}.
5008@end deffn
5009
5010@deffn {Directive} %destructor
5011Specify how the parser should reclaim the memory associated to
5012discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
5013@end deffn
5014
5015@deffn {Directive} %file-prefix "@var{prefix}"
5016Specify a prefix to use for all Bison output file names. The names
5017are chosen as if the grammar file were named @file{@var{prefix}.y}.
5018@end deffn
5019
5020@deffn {Directive} %language "@var{language}"
5021Specify the programming language for the generated parser. Currently
5022supported languages include C, C++, and Java.
5023@var{language} is case-insensitive.
5024
5025This directive is experimental and its effect may be modified in future
5026releases.
5027@end deffn
5028
5029@deffn {Directive} %locations
5030Generate the code processing the locations (@pxref{Action Features,
5031,Special Features for Use in Actions}). This mode is enabled as soon as
5032the grammar uses the special @samp{@@@var{n}} tokens, but if your
5033grammar does not use it, using @samp{%locations} allows for more
5034accurate syntax error messages.
5035@end deffn
5036
5037@deffn {Directive} %name-prefix "@var{prefix}"
5038Rename the external symbols used in the parser so that they start with
5039@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
5040in C parsers
5041is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5042@code{yylval}, @code{yychar}, @code{yydebug}, and
5043(if locations are used) @code{yylloc}. If you use a push parser,
5044@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5045@code{yypstate_new} and @code{yypstate_delete} will
5046also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
5047names become @code{c_parse}, @code{c_lex}, and so on.
5048For C++ parsers, see the @samp{%define api.namespace} documentation in this
5049section.
5050@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
5051@end deffn
5052
5053@ifset defaultprec
5054@deffn {Directive} %no-default-prec
5055Do not assign a precedence to rules lacking an explicit @code{%prec}
5056modifier (@pxref{Contextual Precedence, ,Context-Dependent
5057Precedence}).
5058@end deffn
5059@end ifset
5060
5061@deffn {Directive} %no-lines
5062Don't generate any @code{#line} preprocessor commands in the parser
5063implementation file. Ordinarily Bison writes these commands in the
5064parser implementation file so that the C compiler and debuggers will
5065associate errors and object code with your source file (the grammar
5066file). This directive causes them to associate errors with the parser
5067implementation file, treating it as an independent source file in its
5068own right.
5069@end deffn
5070
5071@deffn {Directive} %output "@var{file}"
5072Specify @var{file} for the parser implementation file.
5073@end deffn
5074
5075@deffn {Directive} %pure-parser
5076Deprecated version of @samp{%define api.pure} (@pxref{%define
5077Summary,,api.pure}), for which Bison is more careful to warn about
5078unreasonable usage.
5079@end deffn
5080
5081@deffn {Directive} %require "@var{version}"
5082Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5083Require a Version of Bison}.
5084@end deffn
5085
5086@deffn {Directive} %skeleton "@var{file}"
5087Specify the skeleton to use.
5088
5089@c You probably don't need this option unless you are developing Bison.
5090@c You should use @code{%language} if you want to specify the skeleton for a
5091@c different language, because it is clearer and because it will always choose the
5092@c correct skeleton for non-deterministic or push parsers.
5093
5094If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5095file in the Bison installation directory.
5096If it does, @var{file} is an absolute file name or a file name relative to the
5097directory of the grammar file.
5098This is similar to how most shells resolve commands.
5099@end deffn
5100
5101@deffn {Directive} %token-table
5102Generate an array of token names in the parser implementation file.
5103The name of the array is @code{yytname}; @code{yytname[@var{i}]} is
5104the name of the token whose internal Bison token code number is
5105@var{i}. The first three elements of @code{yytname} correspond to the
5106predefined tokens @code{"$end"}, @code{"error"}, and
5107@code{"$undefined"}; after these come the symbols defined in the
5108grammar file.
5109
5110The name in the table includes all the characters needed to represent
5111the token in Bison. For single-character literals and literal
5112strings, this includes the surrounding quoting characters and any
5113escape sequences. For example, the Bison single-character literal
5114@code{'+'} corresponds to a three-character name, represented in C as
5115@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5116corresponds to a five-character name, represented in C as
5117@code{"\"\\\\/\""}.
5118
5119When you specify @code{%token-table}, Bison also generates macro
5120definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5121@code{YYNRULES}, and @code{YYNSTATES}:
5122
5123@table @code
5124@item YYNTOKENS
5125The highest token number, plus one.
5126@item YYNNTS
5127The number of nonterminal symbols.
5128@item YYNRULES
5129The number of grammar rules,
5130@item YYNSTATES
5131The number of parser states (@pxref{Parser States}).
5132@end table
5133@end deffn
5134
5135@deffn {Directive} %verbose
5136Write an extra output file containing verbose descriptions of the
5137parser states and what is done for each type of lookahead token in
5138that state. @xref{Understanding, , Understanding Your Parser}, for more
5139information.
5140@end deffn
5141
5142@deffn {Directive} %yacc
5143Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5144including its naming conventions. @xref{Bison Options}, for more.
5145@end deffn
5146
5147
5148@node %define Summary
5149@subsection %define Summary
51151d91
JD
5150
5151There are many features of Bison's behavior that can be controlled by
5152assigning the feature a single value. For historical reasons, some
5153such features are assigned values by dedicated directives, such as
5154@code{%start}, which assigns the start symbol. However, newer such
5155features are associated with variables, which are assigned by the
5156@code{%define} directive:
5157
c1d19e10 5158@deffn {Directive} %define @var{variable}
cf499cff 5159@deffnx {Directive} %define @var{variable} @var{value}
c1d19e10 5160@deffnx {Directive} %define @var{variable} "@var{value}"
51151d91 5161Define @var{variable} to @var{value}.
9611cfa2 5162
51151d91
JD
5163@var{value} must be placed in quotation marks if it contains any
5164character other than a letter, underscore, period, or non-initial dash
5165or digit. Omitting @code{"@var{value}"} entirely is always equivalent
5166to specifying @code{""}.
9611cfa2 5167
51151d91
JD
5168It is an error if a @var{variable} is defined by @code{%define}
5169multiple times, but see @ref{Bison Options,,-D
5170@var{name}[=@var{value}]}.
5171@end deffn
cf499cff 5172
51151d91
JD
5173The rest of this section summarizes variables and values that
5174@code{%define} accepts.
9611cfa2 5175
51151d91
JD
5176Some @var{variable}s take Boolean values. In this case, Bison will
5177complain if the variable definition does not meet one of the following
5178four conditions:
9611cfa2
JD
5179
5180@enumerate
cf499cff 5181@item @code{@var{value}} is @code{true}
9611cfa2 5182
cf499cff
JD
5183@item @code{@var{value}} is omitted (or @code{""} is specified).
5184This is equivalent to @code{true}.
9611cfa2 5185
cf499cff 5186@item @code{@var{value}} is @code{false}.
9611cfa2
JD
5187
5188@item @var{variable} is never defined.
c6abeab1 5189In this case, Bison selects a default value.
9611cfa2 5190@end enumerate
148d66d8 5191
c6abeab1
JD
5192What @var{variable}s are accepted, as well as their meanings and default
5193values, depend on the selected target language and/or the parser
5194skeleton (@pxref{Decl Summary,,%language}, @pxref{Decl
5195Summary,,%skeleton}).
5196Unaccepted @var{variable}s produce an error.
793fbca5
JD
5197Some of the accepted @var{variable}s are:
5198
fa819509 5199@table @code
6b5a0de9 5200@c ================================================== api.namespace
67501061
AD
5201@item api.namespace
5202@findex %define api.namespace
5203@itemize
5204@item Languages(s): C++
5205
f1b238df 5206@item Purpose: Specify the namespace for the parser class.
67501061
AD
5207For example, if you specify:
5208
5209@smallexample
5210%define api.namespace "foo::bar"
5211@end smallexample
5212
5213Bison uses @code{foo::bar} verbatim in references such as:
5214
5215@smallexample
5216foo::bar::parser::semantic_type
5217@end smallexample
5218
5219However, to open a namespace, Bison removes any leading @code{::} and then
5220splits on any remaining occurrences:
5221
5222@smallexample
5223namespace foo @{ namespace bar @{
5224 class position;
5225 class location;
5226@} @}
5227@end smallexample
5228
5229@item Accepted Values:
5230Any absolute or relative C++ namespace reference without a trailing
5231@code{"::"}. For example, @code{"foo"} or @code{"::foo::bar"}.
5232
5233@item Default Value:
5234The value specified by @code{%name-prefix}, which defaults to @code{yy}.
5235This usage of @code{%name-prefix} is for backward compatibility and can
5236be confusing since @code{%name-prefix} also specifies the textual prefix
5237for the lexical analyzer function. Thus, if you specify
5238@code{%name-prefix}, it is best to also specify @samp{%define
5239api.namespace} so that @code{%name-prefix} @emph{only} affects the
5240lexical analyzer function. For example, if you specify:
5241
5242@smallexample
5243%define api.namespace "foo"
5244%name-prefix "bar::"
5245@end smallexample
5246
5247The parser namespace is @code{foo} and @code{yylex} is referenced as
5248@code{bar::lex}.
5249@end itemize
5250@c namespace
5251
5252
5253
5254@c ================================================== api.pure
d9df47b6
JD
5255@item api.pure
5256@findex %define api.pure
5257
5258@itemize @bullet
5259@item Language(s): C
5260
5261@item Purpose: Request a pure (reentrant) parser program.
5262@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
5263
5264@item Accepted Values: Boolean
5265
cf499cff 5266@item Default Value: @code{false}
d9df47b6 5267@end itemize
71b00ed8 5268@c api.pure
d9df47b6 5269
67501061
AD
5270
5271
5272@c ================================================== api.push-pull
67212941
JD
5273@item api.push-pull
5274@findex %define api.push-pull
793fbca5
JD
5275
5276@itemize @bullet
eb45ef3b 5277@item Language(s): C (deterministic parsers only)
793fbca5 5278
f1b238df 5279@item Purpose: Request a pull parser, a push parser, or both.
d782395d 5280@xref{Push Decl, ,A Push Parser}.
59da312b
JD
5281(The current push parsing interface is experimental and may evolve.
5282More user feedback will help to stabilize it.)
793fbca5 5283
cf499cff 5284@item Accepted Values: @code{pull}, @code{push}, @code{both}
793fbca5 5285
cf499cff 5286@item Default Value: @code{pull}
793fbca5 5287@end itemize
67212941 5288@c api.push-pull
71b00ed8 5289
6b5a0de9
AD
5290
5291
5292@c ================================================== api.tokens.prefix
4c6622c2
AD
5293@item api.tokens.prefix
5294@findex %define api.tokens.prefix
5295
5296@itemize
5297@item Languages(s): all
5298
5299@item Purpose:
5300Add a prefix to the token names when generating their definition in the
5301target language. For instance
5302
5303@example
5304%token FILE for ERROR
5305%define api.tokens.prefix "TOK_"
5306%%
5307start: FILE for ERROR;
5308@end example
5309
5310@noindent
5311generates the definition of the symbols @code{TOK_FILE}, @code{TOK_for},
5312and @code{TOK_ERROR} in the generated source files. In particular, the
5313scanner must use these prefixed token names, while the grammar itself
5314may still use the short names (as in the sample rule given above). The
5315generated informational files (@file{*.output}, @file{*.xml},
5316@file{*.dot}) are not modified by this prefix. See @ref{Calc++ Parser}
5317and @ref{Calc++ Scanner}, for a complete example.
5318
5319@item Accepted Values:
5320Any string. Should be a valid identifier prefix in the target language,
5321in other words, it should typically be an identifier itself (sequence of
5322letters, underscores, and ---not at the beginning--- digits).
5323
5324@item Default Value:
5325empty
5326@end itemize
5327@c api.tokens.prefix
5328
5329
3cdc21cf 5330@c ================================================== lex_symbol
84072495 5331@item lex_symbol
3cdc21cf
AD
5332@findex %define lex_symbol
5333
5334@itemize @bullet
5335@item Language(s):
5336C++
5337
5338@item Purpose:
5339When variant-based semantic values are enabled (@pxref{C++ Variants}),
5340request that symbols be handled as a whole (type, value, and possibly
5341location) in the scanner. @xref{Complete Symbols}, for details.
5342
5343@item Accepted Values:
5344Boolean.
5345
5346@item Default Value:
5347@code{false}
5348@end itemize
5349@c lex_symbol
5350
5351
6b5a0de9
AD
5352@c ================================================== lr.default-reductions
5353
5bab9d08 5354@item lr.default-reductions
110ef36a 5355@cindex default reductions
5bab9d08 5356@findex %define lr.default-reductions
eb45ef3b
JD
5357@cindex delayed syntax errors
5358@cindex syntax errors delayed
8a4281b9 5359@cindex LAC
fcf834f9 5360@findex %nonassoc
eb45ef3b
JD
5361
5362@itemize @bullet
5363@item Language(s): all
5364
fcf834f9 5365@item Purpose: Specify the kind of states that are permitted to
110ef36a 5366contain default reductions.
fcf834f9
JD
5367That is, in such a state, Bison selects the reduction with the largest
5368lookahead set to be the default parser action and then removes that
110ef36a 5369lookahead set.
fcf834f9
JD
5370(The ability to specify where default reductions should be used is
5371experimental.
eb45ef3b
JD
5372More user feedback will help to stabilize it.)
5373
5374@item Accepted Values:
5375@itemize
cf499cff 5376@item @code{all}.
35c1e5f0
JD
5377This is the traditional Bison behavior. The main advantage is a
5378significant decrease in the size of the parser tables. The
5379disadvantage is that, when the generated parser encounters a
fcf834f9
JD
5380syntactically unacceptable token, the parser might then perform
5381unnecessary default reductions before it can detect the syntax error.
35c1e5f0
JD
5382Such delayed syntax error detection is usually inherent in LALR and
5383IELR parser tables anyway due to LR state merging (@pxref{%define
5384Summary,,lr.type}). Furthermore, the use of @code{%nonassoc} can
5385contribute to delayed syntax error detection even in the case of
5386canonical LR. As an experimental feature, delayed syntax error
5387detection can be overcome in all cases by enabling LAC (@pxref{%define
5388Summary,,parse.lac}, for details, including a discussion of the
5389effects of delayed syntax error detection).
eb45ef3b 5390
cf499cff 5391@item @code{consistent}.
eb45ef3b
JD
5392@cindex consistent states
5393A consistent state is a state that has only one possible action.
5394If that action is a reduction, then the parser does not need to request
5395a lookahead token from the scanner before performing that action.
fcf834f9
JD
5396However, the parser recognizes the ability to ignore the lookahead token
5397in this way only when such a reduction is encoded as a default
5398reduction.
5399Thus, if default reductions are permitted only in consistent states,
8a4281b9 5400then a canonical LR parser that does not employ
fcf834f9
JD
5401@code{%nonassoc} detects a syntax error as soon as it @emph{needs} the
5402syntactically unacceptable token from the scanner.
eb45ef3b 5403
cf499cff 5404@item @code{accepting}.
eb45ef3b 5405@cindex accepting state
fcf834f9
JD
5406In the accepting state, the default reduction is actually the accept
5407action.
8a4281b9 5408In this case, a canonical LR parser that does not employ
fcf834f9
JD
5409@code{%nonassoc} detects a syntax error as soon as it @emph{reaches} the
5410syntactically unacceptable token in the input.
5411That is, it does not perform any extra reductions.
eb45ef3b
JD
5412@end itemize
5413
5414@item Default Value:
5415@itemize
cf499cff
JD
5416@item @code{accepting} if @code{lr.type} is @code{canonical-lr}.
5417@item @code{all} otherwise.
eb45ef3b
JD
5418@end itemize
5419@end itemize
5420
6b5a0de9
AD
5421@c ============================================ lr.keep-unreachable-states
5422
67212941
JD
5423@item lr.keep-unreachable-states
5424@findex %define lr.keep-unreachable-states
31984206
JD
5425
5426@itemize @bullet
5427@item Language(s): all
5428
f1b238df
JD
5429@item Purpose: Request that Bison allow unreachable parser states to
5430remain in the parser tables.
31984206
JD
5431Bison considers a state to be unreachable if there exists no sequence of
5432transitions from the start state to that state.
5433A state can become unreachable during conflict resolution if Bison disables a
5434shift action leading to it from a predecessor state.
5435Keeping unreachable states is sometimes useful for analysis purposes, but they
5436are useless in the generated parser.
5437
5438@item Accepted Values: Boolean
5439
cf499cff 5440@item Default Value: @code{false}
31984206
JD
5441
5442@item Caveats:
5443
5444@itemize @bullet
cff03fb2
JD
5445
5446@item Unreachable states may contain conflicts and may use rules not used in
5447any other state.
31984206
JD
5448Thus, keeping unreachable states may induce warnings that are irrelevant to
5449your parser's behavior, and it may eliminate warnings that are relevant.
5450Of course, the change in warnings may actually be relevant to a parser table
5451analysis that wants to keep unreachable states, so this behavior will likely
5452remain in future Bison releases.
5453
5454@item While Bison is able to remove unreachable states, it is not guaranteed to
5455remove other kinds of useless states.
5456Specifically, when Bison disables reduce actions during conflict resolution,
5457some goto actions may become useless, and thus some additional states may
5458become useless.
5459If Bison were to compute which goto actions were useless and then disable those
5460actions, it could identify such states as unreachable and then remove those
5461states.
5462However, Bison does not compute which goto actions are useless.
5463@end itemize
5464@end itemize
67212941 5465@c lr.keep-unreachable-states
31984206 5466
6b5a0de9
AD
5467@c ================================================== lr.type
5468
eb45ef3b
JD
5469@item lr.type
5470@findex %define lr.type
8a4281b9
JD
5471@cindex LALR
5472@cindex IELR
5473@cindex LR
eb45ef3b
JD
5474
5475@itemize @bullet
5476@item Language(s): all
5477
f1b238df 5478@item Purpose: Specify the type of parser tables within the
8a4281b9 5479LR(1) family.
eb45ef3b
JD
5480(This feature is experimental.
5481More user feedback will help to stabilize it.)
5482
5483@item Accepted Values:
5484@itemize
cf499cff 5485@item @code{lalr}.
8a4281b9
JD
5486While Bison generates LALR parser tables by default for
5487historical reasons, IELR or canonical LR is almost
eb45ef3b 5488always preferable for deterministic parsers.
8a4281b9 5489The trouble is that LALR parser tables can suffer from
110ef36a 5490mysterious conflicts and thus may not accept the full set of sentences
8a4281b9 5491that IELR and canonical LR accept.
eb45ef3b 5492@xref{Mystery Conflicts}, for details.
8a4281b9 5493However, there are at least two scenarios where LALR may be
eb45ef3b
JD
5494worthwhile:
5495@itemize
8a4281b9
JD
5496@cindex GLR with LALR
5497@item When employing GLR parsers (@pxref{GLR Parsers}), if you
eb45ef3b
JD
5498do not resolve any conflicts statically (for example, with @code{%left}
5499or @code{%prec}), then the parser explores all potential parses of any
5500given input.
8a4281b9 5501In this case, the use of LALR parser tables is guaranteed not
110ef36a 5502to alter the language accepted by the parser.
8a4281b9 5503LALR parser tables are the smallest parser tables Bison can
eb45ef3b 5504currently generate, so they may be preferable.
f1b238df 5505Nevertheless, once you begin to resolve conflicts statically,
8a4281b9
JD
5506GLR begins to behave more like a deterministic parser, and so
5507IELR and canonical LR can be helpful to avoid
5508LALR's mysterious behavior.
eb45ef3b
JD
5509
5510@item Occasionally during development, an especially malformed grammar
8a4281b9
JD
5511with a major recurring flaw may severely impede the IELR or
5512canonical LR parser table generation algorithm.
5513LALR can be a quick way to generate parser tables in order to
eb45ef3b 5514investigate such problems while ignoring the more subtle differences
8a4281b9 5515from IELR and canonical LR.
eb45ef3b
JD
5516@end itemize
5517
cf499cff 5518@item @code{ielr}.
8a4281b9
JD
5519IELR is a minimal LR algorithm.
5520That is, given any grammar (LR or non-LR),
5521IELR and canonical LR always accept exactly the same
eb45ef3b 5522set of sentences.
8a4281b9
JD
5523However, as for LALR, the number of parser states is often an
5524order of magnitude less for IELR than for canonical
5525LR.
5526More importantly, because canonical LR's extra parser states
5527may contain duplicate conflicts in the case of non-LR
5528grammars, the number of conflicts for IELR is often an order
eb45ef3b
JD
5529of magnitude less as well.
5530This can significantly reduce the complexity of developing of a grammar.
5531
cf499cff 5532@item @code{canonical-lr}.
eb45ef3b
JD
5533@cindex delayed syntax errors
5534@cindex syntax errors delayed
8a4281b9 5535@cindex LAC
fcf834f9 5536@findex %nonassoc
35c1e5f0
JD
5537While inefficient, canonical LR parser tables can be an interesting
5538means to explore a grammar because they have a property that IELR and
5539LALR tables do not. That is, if @code{%nonassoc} is not used and
5540default reductions are left disabled (@pxref{%define
5541Summary,,lr.default-reductions}), then, for every left context of
5542every canonical LR state, the set of tokens accepted by that state is
5543guaranteed to be the exact set of tokens that is syntactically
5544acceptable in that left context. It might then seem that an advantage
5545of canonical LR parsers in production is that, under the above
5546constraints, they are guaranteed to detect a syntax error as soon as
5547possible without performing any unnecessary reductions. However, IELR
5548parsers using LAC (@pxref{%define Summary,,parse.lac}) are also able
5549to achieve this behavior without sacrificing @code{%nonassoc} or
5550default reductions.
eb45ef3b
JD
5551@end itemize
5552
cf499cff 5553@item Default Value: @code{lalr}
eb45ef3b
JD
5554@end itemize
5555
67501061
AD
5556
5557@c ================================================== namespace
793fbca5
JD
5558@item namespace
5559@findex %define namespace
67501061 5560Obsoleted by @code{api.namespace}
fa819509
AD
5561@c namespace
5562
31b850d2
AD
5563
5564@c ================================================== parse.assert
0c90a1f5
AD
5565@item parse.assert
5566@findex %define parse.assert
5567
5568@itemize
5569@item Languages(s): C++
5570
5571@item Purpose: Issue runtime assertions to catch invalid uses.
3cdc21cf
AD
5572In C++, when variants are used (@pxref{C++ Variants}), symbols must be
5573constructed and
0c90a1f5
AD
5574destroyed properly. This option checks these constraints.
5575
5576@item Accepted Values: Boolean
5577
5578@item Default Value: @code{false}
5579@end itemize
5580@c parse.assert
5581
31b850d2
AD
5582
5583@c ================================================== parse.error
5584@item parse.error
5585@findex %define parse.error
5586@itemize
5587@item Languages(s):
fcf834f9 5588all
31b850d2
AD
5589@item Purpose:
5590Control the kind of error messages passed to the error reporting
5591function. @xref{Error Reporting, ,The Error Reporting Function
5592@code{yyerror}}.
5593@item Accepted Values:
5594@itemize
cf499cff 5595@item @code{simple}
31b850d2
AD
5596Error messages passed to @code{yyerror} are simply @w{@code{"syntax
5597error"}}.
cf499cff 5598@item @code{verbose}
31b850d2
AD
5599Error messages report the unexpected token, and possibly the expected
5600ones.
5601@end itemize
5602
5603@item Default Value:
5604@code{simple}
5605@end itemize
5606@c parse.error
5607
5608
fcf834f9
JD
5609@c ================================================== parse.lac
5610@item parse.lac
5611@findex %define parse.lac
8a4281b9 5612@cindex LAC
fcf834f9
JD
5613@cindex lookahead correction
5614
5615@itemize
5616@item Languages(s): C
5617
8a4281b9 5618@item Purpose: Enable LAC (lookahead correction) to improve
fcf834f9
JD
5619syntax error handling.
5620
8a4281b9 5621Canonical LR, IELR, and LALR can suffer
fcf834f9
JD
5622from a couple of problems upon encountering a syntax error. First, the
5623parser might perform additional parser stack reductions before
5624discovering the syntax error. Such reductions perform user semantic
5625actions that are unexpected because they are based on an invalid token,
5626and they cause error recovery to begin in a different syntactic context
5627than the one in which the invalid token was encountered. Second, when
5628verbose error messages are enabled (with @code{%error-verbose} or
5629@code{#define YYERROR_VERBOSE}), the expected token list in the syntax
5630error message can both contain invalid tokens and omit valid tokens.
5631
5632The culprits for the above problems are @code{%nonassoc}, default
5633reductions in inconsistent states, and parser state merging. Thus,
8a4281b9
JD
5634IELR and LALR suffer the most. Canonical
5635LR can suffer only if @code{%nonassoc} is used or if default
fcf834f9
JD
5636reductions are enabled for inconsistent states.
5637
8a4281b9
JD
5638LAC is a new mechanism within the parsing algorithm that
5639completely solves these problems for canonical LR,
5640IELR, and LALR without sacrificing @code{%nonassoc},
fcf834f9
JD
5641default reductions, or state mering. Conceptually, the mechanism is
5642straight-forward. Whenever the parser fetches a new token from the
5643scanner so that it can determine the next parser action, it immediately
5644suspends normal parsing and performs an exploratory parse using a
5645temporary copy of the normal parser state stack. During this
5646exploratory parse, the parser does not perform user semantic actions.
5647If the exploratory parse reaches a shift action, normal parsing then
5648resumes on the normal parser stacks. If the exploratory parse reaches
5649an error instead, the parser reports a syntax error. If verbose syntax
5650error messages are enabled, the parser must then discover the list of
5651expected tokens, so it performs a separate exploratory parse for each
5652token in the grammar.
5653
35c1e5f0
JD
5654There is one subtlety about the use of LAC. That is, when in a
5655consistent parser state with a default reduction, the parser will not
5656attempt to fetch a token from the scanner because no lookahead is
5657needed to determine the next parser action. Thus, whether default
5658reductions are enabled in consistent states (@pxref{%define
fcf834f9
JD
5659Summary,,lr.default-reductions}) affects how soon the parser detects a
5660syntax error: when it @emph{reaches} an erroneous token or when it
35c1e5f0
JD
5661eventually @emph{needs} that token as a lookahead. The latter
5662behavior is probably more intuitive, so Bison currently provides no
5663way to achieve the former behavior while default reductions are fully
5664enabled.
fcf834f9 5665
8a4281b9 5666Thus, when LAC is in use, for some fixed decision of whether
fcf834f9 5667to enable default reductions in consistent states, canonical
8a4281b9 5668LR and IELR behave exactly the same for both
fcf834f9 5669syntactically acceptable and syntactically unacceptable input. While
8a4281b9
JD
5670LALR still does not support the full language-recognition
5671power of canonical LR and IELR, LAC at
5672least enables LALR's syntax error handling to correctly
5673reflect LALR's language-recognition power.
fcf834f9 5674
8a4281b9 5675Because LAC requires many parse actions to be performed twice,
fcf834f9
JD
5676it can have a performance penalty. However, not all parse actions must
5677be performed twice. Specifically, during a series of default reductions
5678in consistent states and shift actions, the parser never has to initiate
5679an exploratory parse. Moreover, the most time-consuming tasks in a
5680parse are often the file I/O, the lexical analysis performed by the
5681scanner, and the user's semantic actions, but none of these are
5682performed during the exploratory parse. Finally, the base of the
5683temporary stack used during an exploratory parse is a pointer into the
5684normal parser state stack so that the stack is never physically copied.
8a4281b9 5685In our experience, the performance penalty of LAC has proven
fcf834f9
JD
5686insignificant for practical grammars.
5687
5688@item Accepted Values: @code{none}, @code{full}
5689
5690@item Default Value: @code{none}
5691@end itemize
5692@c parse.lac
5693
31b850d2 5694@c ================================================== parse.trace
fa819509
AD
5695@item parse.trace
5696@findex %define parse.trace
5697
5698@itemize
5699@item Languages(s): C, C++
5700
5701@item Purpose: Require parser instrumentation for tracing.
ff7571c0
JD
5702In C/C++, define the macro @code{YYDEBUG} to 1 in the parser implementation
5703file if it is not already defined, so that the debugging facilities are
5704compiled. @xref{Tracing, ,Tracing Your Parser}.
793fbca5 5705
fa819509
AD
5706@item Accepted Values: Boolean
5707
5708@item Default Value: @code{false}
5709@end itemize
fa819509 5710@c parse.trace
99c08fb6 5711
3cdc21cf
AD
5712@c ================================================== variant
5713@item variant
5714@findex %define variant
5715
5716@itemize @bullet
5717@item Language(s):
5718C++
5719
5720@item Purpose:
f1b238df 5721Request variant-based semantic values.
3cdc21cf
AD
5722@xref{C++ Variants}.
5723
5724@item Accepted Values:
5725Boolean.
5726
5727@item Default Value:
5728@code{false}
5729@end itemize
5730@c variant
99c08fb6 5731@end table
592d0b1e 5732
d8988b2f 5733
e0c07222
JD
5734@node %code Summary
5735@subsection %code Summary
e0c07222 5736@findex %code
e0c07222 5737@cindex Prologue
51151d91
JD
5738
5739The @code{%code} directive inserts code verbatim into the output
5740parser source at any of a predefined set of locations. It thus serves
5741as a flexible and user-friendly alternative to the traditional Yacc
5742prologue, @code{%@{@var{code}%@}}. This section summarizes the
5743functionality of @code{%code} for the various target languages
5744supported by Bison. For a detailed discussion of how to use
5745@code{%code} in place of @code{%@{@var{code}%@}} for C/C++ and why it
5746is advantageous to do so, @pxref{Prologue Alternatives}.
5747
5748@deffn {Directive} %code @{@var{code}@}
5749This is the unqualified form of the @code{%code} directive. It
5750inserts @var{code} verbatim at a language-dependent default location
5751in the parser implementation.
5752
e0c07222 5753For C/C++, the default location is the parser implementation file
51151d91
JD
5754after the usual contents of the parser header file. Thus, the
5755unqualified form replaces @code{%@{@var{code}%@}} for most purposes.
e0c07222
JD
5756
5757For Java, the default location is inside the parser class.
5758@end deffn
5759
5760@deffn {Directive} %code @var{qualifier} @{@var{code}@}
5761This is the qualified form of the @code{%code} directive.
51151d91
JD
5762@var{qualifier} identifies the purpose of @var{code} and thus the
5763location(s) where Bison should insert it. That is, if you need to
5764specify location-sensitive @var{code} that does not belong at the
5765default location selected by the unqualified @code{%code} form, use
5766this form instead.
5767@end deffn
5768
5769For any particular qualifier or for the unqualified form, if there are
5770multiple occurrences of the @code{%code} directive, Bison concatenates
5771the specified code in the order in which it appears in the grammar
5772file.
e0c07222 5773
51151d91
JD
5774Not all qualifiers are accepted for all target languages. Unaccepted
5775qualifiers produce an error. Some of the accepted qualifiers are:
e0c07222 5776
84072495 5777@table @code
e0c07222
JD
5778@item requires
5779@findex %code requires
5780
5781@itemize @bullet
5782@item Language(s): C, C++
5783
5784@item Purpose: This is the best place to write dependency code required for
5785@code{YYSTYPE} and @code{YYLTYPE}.
5786In other words, it's the best place to define types referenced in @code{%union}
5787directives, and it's the best place to override Bison's default @code{YYSTYPE}
5788and @code{YYLTYPE} definitions.
5789
5790@item Location(s): The parser header file and the parser implementation file
5791before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
5792definitions.
5793@end itemize
5794
5795@item provides
5796@findex %code provides
5797
5798@itemize @bullet
5799@item Language(s): C, C++
5800
5801@item Purpose: This is the best place to write additional definitions and
5802declarations that should be provided to other modules.
5803
5804@item Location(s): The parser header file and the parser implementation
5805file after the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and
5806token definitions.
5807@end itemize
5808
5809@item top
5810@findex %code top
5811
5812@itemize @bullet
5813@item Language(s): C, C++
5814
5815@item Purpose: The unqualified @code{%code} or @code{%code requires}
5816should usually be more appropriate than @code{%code top}. However,
5817occasionally it is necessary to insert code much nearer the top of the
5818parser implementation file. For example:
5819
5820@smallexample
5821%code top @{
5822 #define _GNU_SOURCE
5823 #include <stdio.h>
5824@}
5825@end smallexample
5826
5827@item Location(s): Near the top of the parser implementation file.
5828@end itemize
5829
5830@item imports
5831@findex %code imports
5832
5833@itemize @bullet
5834@item Language(s): Java
5835
5836@item Purpose: This is the best place to write Java import directives.
5837
5838@item Location(s): The parser Java file after any Java package directive and
5839before any class definitions.
5840@end itemize
84072495 5841@end table
e0c07222 5842
51151d91
JD
5843Though we say the insertion locations are language-dependent, they are
5844technically skeleton-dependent. Writers of non-standard skeletons
5845however should choose their locations consistently with the behavior
5846of the standard Bison skeletons.
e0c07222 5847
d8988b2f 5848
342b8b6e 5849@node Multiple Parsers
bfa74976
RS
5850@section Multiple Parsers in the Same Program
5851
5852Most programs that use Bison parse only one language and therefore contain
5853only one Bison parser. But what if you want to parse more than one
5854language with the same program? Then you need to avoid a name conflict
5855between different definitions of @code{yyparse}, @code{yylval}, and so on.
5856
5857The easy way to do this is to use the option @samp{-p @var{prefix}}
704a47c4
AD
5858(@pxref{Invocation, ,Invoking Bison}). This renames the interface
5859functions and variables of the Bison parser to start with @var{prefix}
5860instead of @samp{yy}. You can use this to give each parser distinct
5861names that do not conflict.
bfa74976
RS
5862
5863The precise list of symbols renamed is @code{yyparse}, @code{yylex},
2a8d363a 5864@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
f4101aa6
AD
5865@code{yychar} and @code{yydebug}. If you use a push parser,
5866@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
9987d1b3 5867@code{yypstate_new} and @code{yypstate_delete} will also be renamed.
f4101aa6 5868For example, if you use @samp{-p c}, the names become @code{cparse},
9987d1b3 5869@code{clex}, and so on.
bfa74976
RS
5870
5871@strong{All the other variables and macros associated with Bison are not
5872renamed.} These others are not global; there is no conflict if the same
5873name is used in different parsers. For example, @code{YYSTYPE} is not
5874renamed, but defining this in different ways in different parsers causes
5875no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5876
ff7571c0
JD
5877The @samp{-p} option works by adding macro definitions to the
5878beginning of the parser implementation file, defining @code{yyparse}
5879as @code{@var{prefix}parse}, and so on. This effectively substitutes
5880one name for the other in the entire parser implementation file.
bfa74976 5881
342b8b6e 5882@node Interface
bfa74976
RS
5883@chapter Parser C-Language Interface
5884@cindex C-language interface
5885@cindex interface
5886
5887The Bison parser is actually a C function named @code{yyparse}. Here we
5888describe the interface conventions of @code{yyparse} and the other
5889functions that it needs to use.
5890
5891Keep in mind that the parser uses many C identifiers starting with
5892@samp{yy} and @samp{YY} for internal purposes. If you use such an
75f5aaea
MA
5893identifier (aside from those in this manual) in an action or in epilogue
5894in the grammar file, you are likely to run into trouble.
bfa74976
RS
5895
5896@menu
f5f419de
DJ
5897* Parser Function:: How to call @code{yyparse} and what it returns.
5898* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5899* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
5900* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
5901* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
5902* Lexical:: You must supply a function @code{yylex}
5903 which reads tokens.
5904* Error Reporting:: You must supply a function @code{yyerror}.
5905* Action Features:: Special features for use in actions.
5906* Internationalization:: How to let the parser speak in the user's
5907 native language.
bfa74976
RS
5908@end menu
5909
342b8b6e 5910@node Parser Function
bfa74976
RS
5911@section The Parser Function @code{yyparse}
5912@findex yyparse
5913
5914You call the function @code{yyparse} to cause parsing to occur. This
5915function reads tokens, executes actions, and ultimately returns when it
5916encounters end-of-input or an unrecoverable syntax error. You can also
14ded682
AD
5917write an action which directs @code{yyparse} to return immediately
5918without reading further.
bfa74976 5919
2a8d363a
AD
5920
5921@deftypefun int yyparse (void)
bfa74976
RS
5922The value returned by @code{yyparse} is 0 if parsing was successful (return
5923is due to end-of-input).
5924
b47dbebe
PE
5925The value is 1 if parsing failed because of invalid input, i.e., input
5926that contains a syntax error or that causes @code{YYABORT} to be
5927invoked.
5928
5929The value is 2 if parsing failed due to memory exhaustion.
2a8d363a 5930@end deftypefun
bfa74976
RS
5931
5932In an action, you can cause immediate return from @code{yyparse} by using
5933these macros:
5934
2a8d363a 5935@defmac YYACCEPT
bfa74976
RS
5936@findex YYACCEPT
5937Return immediately with value 0 (to report success).
2a8d363a 5938@end defmac
bfa74976 5939
2a8d363a 5940@defmac YYABORT
bfa74976
RS
5941@findex YYABORT
5942Return immediately with value 1 (to report failure).
2a8d363a
AD
5943@end defmac
5944
5945If you use a reentrant parser, you can optionally pass additional
5946parameter information to it in a reentrant way. To do so, use the
5947declaration @code{%parse-param}:
5948
2055a44e 5949@deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
2a8d363a 5950@findex %parse-param
2055a44e
AD
5951Declare that one or more
5952@var{argument-declaration} are additional @code{yyparse} arguments.
94175978 5953The @var{argument-declaration} is used when declaring
feeb0eda
PE
5954functions or prototypes. The last identifier in
5955@var{argument-declaration} must be the argument name.
2a8d363a
AD
5956@end deffn
5957
5958Here's an example. Write this in the parser:
5959
5960@example
2055a44e 5961%parse-param @{int *nastiness@} @{int *randomness@}
2a8d363a
AD
5962@end example
5963
5964@noindent
5965Then call the parser like this:
5966
5967@example
5968@{
5969 int nastiness, randomness;
5970 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5971 value = yyparse (&nastiness, &randomness);
5972 @dots{}
5973@}
5974@end example
5975
5976@noindent
5977In the grammar actions, use expressions like this to refer to the data:
5978
5979@example
5980exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5981@end example
5982
9987d1b3
JD
5983@node Push Parser Function
5984@section The Push Parser Function @code{yypush_parse}
5985@findex yypush_parse
5986
59da312b
JD
5987(The current push parsing interface is experimental and may evolve.
5988More user feedback will help to stabilize it.)
5989
f4101aa6 5990You call the function @code{yypush_parse} to parse a single token. This
cf499cff
JD
5991function is available if either the @samp{%define api.push-pull push} or
5992@samp{%define api.push-pull both} declaration is used.
9987d1b3
JD
5993@xref{Push Decl, ,A Push Parser}.
5994
5995@deftypefun int yypush_parse (yypstate *yyps)
f4101aa6 5996The value returned by @code{yypush_parse} is the same as for yyparse with the
9987d1b3
JD
5997following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5998is required to finish parsing the grammar.
5999@end deftypefun
6000
6001@node Pull Parser Function
6002@section The Pull Parser Function @code{yypull_parse}
6003@findex yypull_parse
6004
59da312b
JD
6005(The current push parsing interface is experimental and may evolve.
6006More user feedback will help to stabilize it.)
6007
f4101aa6 6008You call the function @code{yypull_parse} to parse the rest of the input
cf499cff 6009stream. This function is available if the @samp{%define api.push-pull both}
f4101aa6 6010declaration is used.
9987d1b3
JD
6011@xref{Push Decl, ,A Push Parser}.
6012
6013@deftypefun int yypull_parse (yypstate *yyps)
6014The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
6015@end deftypefun
6016
6017@node Parser Create Function
6018@section The Parser Create Function @code{yystate_new}
6019@findex yypstate_new
6020
59da312b
JD
6021(The current push parsing interface is experimental and may evolve.
6022More user feedback will help to stabilize it.)
6023
f4101aa6 6024You call the function @code{yypstate_new} to create a new parser instance.
cf499cff
JD
6025This function is available if either the @samp{%define api.push-pull push} or
6026@samp{%define api.push-pull both} declaration is used.
9987d1b3
JD
6027@xref{Push Decl, ,A Push Parser}.
6028
6029@deftypefun yypstate *yypstate_new (void)
f50bfcd6 6030The function will return a valid parser instance if there was memory available
333e670c
JD
6031or 0 if no memory was available.
6032In impure mode, it will also return 0 if a parser instance is currently
6033allocated.
9987d1b3
JD
6034@end deftypefun
6035
6036@node Parser Delete Function
6037@section The Parser Delete Function @code{yystate_delete}
6038@findex yypstate_delete
6039
59da312b
JD
6040(The current push parsing interface is experimental and may evolve.
6041More user feedback will help to stabilize it.)
6042
9987d1b3 6043You call the function @code{yypstate_delete} to delete a parser instance.
cf499cff
JD
6044function is available if either the @samp{%define api.push-pull push} or
6045@samp{%define api.push-pull both} declaration is used.
9987d1b3
JD
6046@xref{Push Decl, ,A Push Parser}.
6047
6048@deftypefun void yypstate_delete (yypstate *yyps)
6049This function will reclaim the memory associated with a parser instance.
6050After this call, you should no longer attempt to use the parser instance.
6051@end deftypefun
bfa74976 6052
342b8b6e 6053@node Lexical
bfa74976
RS
6054@section The Lexical Analyzer Function @code{yylex}
6055@findex yylex
6056@cindex lexical analyzer
6057
6058The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
6059the input stream and returns them to the parser. Bison does not create
6060this function automatically; you must write it so that @code{yyparse} can
6061call it. The function is sometimes referred to as a lexical scanner.
6062
ff7571c0
JD
6063In simple programs, @code{yylex} is often defined at the end of the
6064Bison grammar file. If @code{yylex} is defined in a separate source
6065file, you need to arrange for the token-type macro definitions to be
6066available there. To do this, use the @samp{-d} option when you run
6067Bison, so that it will write these macro definitions into the separate
6068parser header file, @file{@var{name}.tab.h}, which you can include in
6069the other source files that need it. @xref{Invocation, ,Invoking
6070Bison}.
bfa74976
RS
6071
6072@menu
6073* Calling Convention:: How @code{yyparse} calls @code{yylex}.
f5f419de
DJ
6074* Token Values:: How @code{yylex} must return the semantic value
6075 of the token it has read.
6076* Token Locations:: How @code{yylex} must return the text location
6077 (line number, etc.) of the token, if the
6078 actions want that.
6079* Pure Calling:: How the calling convention differs in a pure parser
6080 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
bfa74976
RS
6081@end menu
6082
342b8b6e 6083@node Calling Convention
bfa74976
RS
6084@subsection Calling Convention for @code{yylex}
6085
72d2299c
PE
6086The value that @code{yylex} returns must be the positive numeric code
6087for the type of token it has just found; a zero or negative value
6088signifies end-of-input.
bfa74976
RS
6089
6090When a token is referred to in the grammar rules by a name, that name
ff7571c0
JD
6091in the parser implementation file becomes a C macro whose definition
6092is the proper numeric code for that token type. So @code{yylex} can
6093use the name to indicate that type. @xref{Symbols}.
bfa74976
RS
6094
6095When a token is referred to in the grammar rules by a character literal,
6096the numeric code for that character is also the code for the token type.
72d2299c
PE
6097So @code{yylex} can simply return that character code, possibly converted
6098to @code{unsigned char} to avoid sign-extension. The null character
6099must not be used this way, because its code is zero and that
bfa74976
RS
6100signifies end-of-input.
6101
6102Here is an example showing these things:
6103
6104@example
13863333
AD
6105int
6106yylex (void)
bfa74976
RS
6107@{
6108 @dots{}
72d2299c 6109 if (c == EOF) /* Detect end-of-input. */
bfa74976
RS
6110 return 0;
6111 @dots{}
6112 if (c == '+' || c == '-')
72d2299c 6113 return c; /* Assume token type for `+' is '+'. */
bfa74976 6114 @dots{}
72d2299c 6115 return INT; /* Return the type of the token. */
bfa74976
RS
6116 @dots{}
6117@}
6118@end example
6119
6120@noindent
6121This interface has been designed so that the output from the @code{lex}
6122utility can be used without change as the definition of @code{yylex}.
6123
931c7513
RS
6124If the grammar uses literal string tokens, there are two ways that
6125@code{yylex} can determine the token type codes for them:
6126
6127@itemize @bullet
6128@item
6129If the grammar defines symbolic token names as aliases for the
6130literal string tokens, @code{yylex} can use these symbolic names like
6131all others. In this case, the use of the literal string tokens in
6132the grammar file has no effect on @code{yylex}.
6133
6134@item
9ecbd125 6135@code{yylex} can find the multicharacter token in the @code{yytname}
931c7513 6136table. The index of the token in the table is the token type's code.
9ecbd125 6137The name of a multicharacter token is recorded in @code{yytname} with a
931c7513 6138double-quote, the token's characters, and another double-quote. The
9e0876fb
PE
6139token's characters are escaped as necessary to be suitable as input
6140to Bison.
931c7513 6141
9e0876fb
PE
6142Here's code for looking up a multicharacter token in @code{yytname},
6143assuming that the characters of the token are stored in
6144@code{token_buffer}, and assuming that the token does not contain any
6145characters like @samp{"} that require escaping.
931c7513
RS
6146
6147@smallexample
6148for (i = 0; i < YYNTOKENS; i++)
6149 @{
6150 if (yytname[i] != 0
6151 && yytname[i][0] == '"'
68449b3a
PE
6152 && ! strncmp (yytname[i] + 1, token_buffer,
6153 strlen (token_buffer))
931c7513
RS
6154 && yytname[i][strlen (token_buffer) + 1] == '"'
6155 && yytname[i][strlen (token_buffer) + 2] == 0)
6156 break;
6157 @}
6158@end smallexample
6159
6160The @code{yytname} table is generated only if you use the
8c9a50be 6161@code{%token-table} declaration. @xref{Decl Summary}.
931c7513
RS
6162@end itemize
6163
342b8b6e 6164@node Token Values
bfa74976
RS
6165@subsection Semantic Values of Tokens
6166
6167@vindex yylval
9d9b8b70 6168In an ordinary (nonreentrant) parser, the semantic value of the token must
bfa74976
RS
6169be stored into the global variable @code{yylval}. When you are using
6170just one data type for semantic values, @code{yylval} has that type.
6171Thus, if the type is @code{int} (the default), you might write this in
6172@code{yylex}:
6173
6174@example
6175@group
6176 @dots{}
72d2299c
PE
6177 yylval = value; /* Put value onto Bison stack. */
6178 return INT; /* Return the type of the token. */
bfa74976
RS
6179 @dots{}
6180@end group
6181@end example
6182
6183When you are using multiple data types, @code{yylval}'s type is a union
704a47c4
AD
6184made from the @code{%union} declaration (@pxref{Union Decl, ,The
6185Collection of Value Types}). So when you store a token's value, you
6186must use the proper member of the union. If the @code{%union}
6187declaration looks like this:
bfa74976
RS
6188
6189@example
6190@group
6191%union @{
6192 int intval;
6193 double val;
6194 symrec *tptr;
6195@}
6196@end group
6197@end example
6198
6199@noindent
6200then the code in @code{yylex} might look like this:
6201
6202@example
6203@group
6204 @dots{}
72d2299c
PE
6205 yylval.intval = value; /* Put value onto Bison stack. */
6206 return INT; /* Return the type of the token. */
bfa74976
RS
6207 @dots{}
6208@end group
6209@end example
6210
95923bd6
AD
6211@node Token Locations
6212@subsection Textual Locations of Tokens
bfa74976
RS
6213
6214@vindex yylloc
847bf1f5 6215If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
f8e1c9e5
AD
6216Tracking Locations}) in actions to keep track of the textual locations
6217of tokens and groupings, then you must provide this information in
6218@code{yylex}. The function @code{yyparse} expects to find the textual
6219location of a token just parsed in the global variable @code{yylloc}.
6220So @code{yylex} must store the proper data in that variable.
847bf1f5
AD
6221
6222By default, the value of @code{yylloc} is a structure and you need only
89cab50d
AD
6223initialize the members that are going to be used by the actions. The
6224four members are called @code{first_line}, @code{first_column},
6225@code{last_line} and @code{last_column}. Note that the use of this
6226feature makes the parser noticeably slower.
bfa74976
RS
6227
6228@tindex YYLTYPE
6229The data type of @code{yylloc} has the name @code{YYLTYPE}.
6230
342b8b6e 6231@node Pure Calling
c656404a 6232@subsection Calling Conventions for Pure Parsers
bfa74976 6233
67501061 6234When you use the Bison declaration @samp{%define api.pure} to request a
e425e872
RS
6235pure, reentrant parser, the global communication variables @code{yylval}
6236and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
6237Parser}.) In such parsers the two global variables are replaced by
6238pointers passed as arguments to @code{yylex}. You must declare them as
6239shown here, and pass the information back by storing it through those
6240pointers.
bfa74976
RS
6241
6242@example
13863333
AD
6243int
6244yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
bfa74976
RS
6245@{
6246 @dots{}
6247 *lvalp = value; /* Put value onto Bison stack. */
6248 return INT; /* Return the type of the token. */
6249 @dots{}
6250@}
6251@end example
6252
6253If the grammar file does not use the @samp{@@} constructs to refer to
95923bd6 6254textual locations, then the type @code{YYLTYPE} will not be defined. In
bfa74976
RS
6255this case, omit the second argument; @code{yylex} will be called with
6256only one argument.
6257
2055a44e 6258If you wish to pass additional arguments to @code{yylex}, use
2a8d363a 6259@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
2055a44e
AD
6260Function}). To pass additional arguments to both @code{yylex} and
6261@code{yyparse}, use @code{%param}.
e425e872 6262
2055a44e 6263@deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
2a8d363a 6264@findex %lex-param
2055a44e
AD
6265Specify that @var{argument-declaration} are additional @code{yylex} argument
6266declarations. You may pass one or more such declarations, which is
6267equivalent to repeating @code{%lex-param}.
6268@end deffn
6269
6270@deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
6271@findex %param
6272Specify that @var{argument-declaration} are additional
6273@code{yylex}/@code{yyparse} argument declaration. This is equivalent to
6274@samp{%lex-param @{@var{argument-declaration}@} @dots{} %parse-param
6275@{@var{argument-declaration}@} @dots{}}. You may pass one or more
6276declarations, which is equivalent to repeating @code{%param}.
2a8d363a 6277@end deffn
e425e872 6278
2a8d363a 6279For instance:
e425e872
RS
6280
6281@example
2055a44e
AD
6282%lex-param @{scanner_mode *mode@}
6283%parse-param @{parser_mode *mode@}
6284%param @{environment_type *env@}
e425e872
RS
6285@end example
6286
6287@noindent
2a8d363a 6288results in the following signature:
e425e872
RS
6289
6290@example
2055a44e
AD
6291int yylex (scanner_mode *mode, environment_type *env);
6292int yyparse (parser_mode *mode, environment_type *env);
e425e872
RS
6293@end example
6294
67501061 6295If @samp{%define api.pure} is added:
c656404a
RS
6296
6297@example
2055a44e
AD
6298int yylex (YYSTYPE *lvalp, scanner_mode *mode, environment_type *env);
6299int yyparse (parser_mode *mode, environment_type *env);
c656404a
RS
6300@end example
6301
2a8d363a 6302@noindent
67501061 6303and finally, if both @samp{%define api.pure} and @code{%locations} are used:
c656404a 6304
2a8d363a 6305@example
2055a44e
AD
6306int yylex (YYSTYPE *lvalp, YYLTYPE *llocp,
6307 scanner_mode *mode, environment_type *env);
6308int yyparse (parser_mode *mode, environment_type *env);
2a8d363a 6309@end example
931c7513 6310
342b8b6e 6311@node Error Reporting
bfa74976
RS
6312@section The Error Reporting Function @code{yyerror}
6313@cindex error reporting function
6314@findex yyerror
6315@cindex parse error
6316@cindex syntax error
6317
31b850d2 6318The Bison parser detects a @dfn{syntax error} (or @dfn{parse error})
9ecbd125 6319whenever it reads a token which cannot satisfy any syntax rule. An
bfa74976 6320action in the grammar can also explicitly proclaim an error, using the
ceed8467
AD
6321macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
6322in Actions}).
bfa74976
RS
6323
6324The Bison parser expects to report the error by calling an error
6325reporting function named @code{yyerror}, which you must supply. It is
6326called by @code{yyparse} whenever a syntax error is found, and it
6e649e65
PE
6327receives one argument. For a syntax error, the string is normally
6328@w{@code{"syntax error"}}.
bfa74976 6329
31b850d2 6330@findex %define parse.error
cf499cff 6331If you invoke @samp{%define parse.error verbose} in the Bison
2a8d363a
AD
6332declarations section (@pxref{Bison Declarations, ,The Bison Declarations
6333Section}), then Bison provides a more verbose and specific error message
6e649e65 6334string instead of just plain @w{@code{"syntax error"}}.
bfa74976 6335
1a059451
PE
6336The parser can detect one other kind of error: memory exhaustion. This
6337can happen when the input contains constructions that are very deeply
bfa74976 6338nested. It isn't likely you will encounter this, since the Bison
1a059451
PE
6339parser normally extends its stack automatically up to a very large limit. But
6340if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
6341fashion, except that the argument string is @w{@code{"memory exhausted"}}.
6342
6343In some cases diagnostics like @w{@code{"syntax error"}} are
6344translated automatically from English to some other language before
6345they are passed to @code{yyerror}. @xref{Internationalization}.
bfa74976
RS
6346
6347The following definition suffices in simple programs:
6348
6349@example
6350@group
13863333 6351void
38a92d50 6352yyerror (char const *s)
bfa74976
RS
6353@{
6354@end group
6355@group
6356 fprintf (stderr, "%s\n", s);
6357@}
6358@end group
6359@end example
6360
6361After @code{yyerror} returns to @code{yyparse}, the latter will attempt
6362error recovery if you have written suitable error recovery grammar rules
6363(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
6364immediately return 1.
6365
93724f13 6366Obviously, in location tracking pure parsers, @code{yyerror} should have
fa7e68c3 6367an access to the current location.
8a4281b9 6368This is indeed the case for the GLR
2a8d363a 6369parsers, but not for the Yacc parser, for historical reasons. I.e., if
d9df47b6 6370@samp{%locations %define api.pure} is passed then the prototypes for
2a8d363a
AD
6371@code{yyerror} are:
6372
6373@example
38a92d50
PE
6374void yyerror (char const *msg); /* Yacc parsers. */
6375void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
2a8d363a
AD
6376@end example
6377
feeb0eda 6378If @samp{%parse-param @{int *nastiness@}} is used, then:
2a8d363a
AD
6379
6380@example
b317297e
PE
6381void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
6382void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
2a8d363a
AD
6383@end example
6384
8a4281b9 6385Finally, GLR and Yacc parsers share the same @code{yyerror} calling
2a8d363a
AD
6386convention for absolutely pure parsers, i.e., when the calling
6387convention of @code{yylex} @emph{and} the calling convention of
67501061 6388@samp{%define api.pure} are pure.
d9df47b6 6389I.e.:
2a8d363a
AD
6390
6391@example
6392/* Location tracking. */
6393%locations
6394/* Pure yylex. */
d9df47b6 6395%define api.pure
feeb0eda 6396%lex-param @{int *nastiness@}
2a8d363a 6397/* Pure yyparse. */
feeb0eda
PE
6398%parse-param @{int *nastiness@}
6399%parse-param @{int *randomness@}
2a8d363a
AD
6400@end example
6401
6402@noindent
6403results in the following signatures for all the parser kinds:
6404
6405@example
6406int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
6407int yyparse (int *nastiness, int *randomness);
93724f13
AD
6408void yyerror (YYLTYPE *locp,
6409 int *nastiness, int *randomness,
38a92d50 6410 char const *msg);
2a8d363a
AD
6411@end example
6412
1c0c3e95 6413@noindent
38a92d50
PE
6414The prototypes are only indications of how the code produced by Bison
6415uses @code{yyerror}. Bison-generated code always ignores the returned
6416value, so @code{yyerror} can return any type, including @code{void}.
6417Also, @code{yyerror} can be a variadic function; that is why the
6418message is always passed last.
6419
6420Traditionally @code{yyerror} returns an @code{int} that is always
6421ignored, but this is purely for historical reasons, and @code{void} is
6422preferable since it more accurately describes the return type for
6423@code{yyerror}.
93724f13 6424
bfa74976
RS
6425@vindex yynerrs
6426The variable @code{yynerrs} contains the number of syntax errors
8a2800e7 6427reported so far. Normally this variable is global; but if you
704a47c4
AD
6428request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
6429then it is a local variable which only the actions can access.
bfa74976 6430
342b8b6e 6431@node Action Features
bfa74976
RS
6432@section Special Features for Use in Actions
6433@cindex summary, action features
6434@cindex action features summary
6435
6436Here is a table of Bison constructs, variables and macros that
6437are useful in actions.
6438
18b519c0 6439@deffn {Variable} $$
bfa74976
RS
6440Acts like a variable that contains the semantic value for the
6441grouping made by the current rule. @xref{Actions}.
18b519c0 6442@end deffn
bfa74976 6443
18b519c0 6444@deffn {Variable} $@var{n}
bfa74976
RS
6445Acts like a variable that contains the semantic value for the
6446@var{n}th component of the current rule. @xref{Actions}.
18b519c0 6447@end deffn
bfa74976 6448
18b519c0 6449@deffn {Variable} $<@var{typealt}>$
bfa74976 6450Like @code{$$} but specifies alternative @var{typealt} in the union
704a47c4
AD
6451specified by the @code{%union} declaration. @xref{Action Types, ,Data
6452Types of Values in Actions}.
18b519c0 6453@end deffn
bfa74976 6454
18b519c0 6455@deffn {Variable} $<@var{typealt}>@var{n}
bfa74976 6456Like @code{$@var{n}} but specifies alternative @var{typealt} in the
13863333 6457union specified by the @code{%union} declaration.
e0c471a9 6458@xref{Action Types, ,Data Types of Values in Actions}.
18b519c0 6459@end deffn
bfa74976 6460
18b519c0 6461@deffn {Macro} YYABORT;
bfa74976
RS
6462Return immediately from @code{yyparse}, indicating failure.
6463@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 6464@end deffn
bfa74976 6465
18b519c0 6466@deffn {Macro} YYACCEPT;
bfa74976
RS
6467Return immediately from @code{yyparse}, indicating success.
6468@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 6469@end deffn
bfa74976 6470
18b519c0 6471@deffn {Macro} YYBACKUP (@var{token}, @var{value});
bfa74976
RS
6472@findex YYBACKUP
6473Unshift a token. This macro is allowed only for rules that reduce
742e4900 6474a single value, and only when there is no lookahead token.
8a4281b9 6475It is also disallowed in GLR parsers.
742e4900 6476It installs a lookahead token with token type @var{token} and
bfa74976
RS
6477semantic value @var{value}; then it discards the value that was
6478going to be reduced by this rule.
6479
6480If the macro is used when it is not valid, such as when there is
742e4900 6481a lookahead token already, then it reports a syntax error with
bfa74976
RS
6482a message @samp{cannot back up} and performs ordinary error
6483recovery.
6484
6485In either case, the rest of the action is not executed.
18b519c0 6486@end deffn
bfa74976 6487
18b519c0 6488@deffn {Macro} YYEMPTY
bfa74976 6489@vindex YYEMPTY
742e4900 6490Value stored in @code{yychar} when there is no lookahead token.
18b519c0 6491@end deffn
bfa74976 6492
32c29292
JD
6493@deffn {Macro} YYEOF
6494@vindex YYEOF
742e4900 6495Value stored in @code{yychar} when the lookahead is the end of the input
32c29292
JD
6496stream.
6497@end deffn
6498
18b519c0 6499@deffn {Macro} YYERROR;
bfa74976
RS
6500@findex YYERROR
6501Cause an immediate syntax error. This statement initiates error
6502recovery just as if the parser itself had detected an error; however, it
6503does not call @code{yyerror}, and does not print any message. If you
6504want to print an error message, call @code{yyerror} explicitly before
6505the @samp{YYERROR;} statement. @xref{Error Recovery}.
18b519c0 6506@end deffn
bfa74976 6507
18b519c0 6508@deffn {Macro} YYRECOVERING
02103984
PE
6509@findex YYRECOVERING
6510The expression @code{YYRECOVERING ()} yields 1 when the parser
6511is recovering from a syntax error, and 0 otherwise.
bfa74976 6512@xref{Error Recovery}.
18b519c0 6513@end deffn
bfa74976 6514
18b519c0 6515@deffn {Variable} yychar
742e4900
JD
6516Variable containing either the lookahead token, or @code{YYEOF} when the
6517lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
32c29292
JD
6518has been performed so the next token is not yet known.
6519Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
6520Actions}).
742e4900 6521@xref{Lookahead, ,Lookahead Tokens}.
18b519c0 6522@end deffn
bfa74976 6523
18b519c0 6524@deffn {Macro} yyclearin;
742e4900 6525Discard the current lookahead token. This is useful primarily in
32c29292
JD
6526error rules.
6527Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
6528Semantic Actions}).
6529@xref{Error Recovery}.
18b519c0 6530@end deffn
bfa74976 6531
18b519c0 6532@deffn {Macro} yyerrok;
bfa74976 6533Resume generating error messages immediately for subsequent syntax
13863333 6534errors. This is useful primarily in error rules.
bfa74976 6535@xref{Error Recovery}.
18b519c0 6536@end deffn
bfa74976 6537
32c29292 6538@deffn {Variable} yylloc
742e4900 6539Variable containing the lookahead token location when @code{yychar} is not set
32c29292
JD
6540to @code{YYEMPTY} or @code{YYEOF}.
6541Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
6542Actions}).
6543@xref{Actions and Locations, ,Actions and Locations}.
6544@end deffn
6545
6546@deffn {Variable} yylval
742e4900 6547Variable containing the lookahead token semantic value when @code{yychar} is
32c29292
JD
6548not set to @code{YYEMPTY} or @code{YYEOF}.
6549Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
6550Actions}).
6551@xref{Actions, ,Actions}.
6552@end deffn
6553
18b519c0 6554@deffn {Value} @@$
847bf1f5 6555@findex @@$
95923bd6 6556Acts like a structure variable containing information on the textual location
847bf1f5
AD
6557of the grouping made by the current rule. @xref{Locations, ,
6558Tracking Locations}.
bfa74976 6559
847bf1f5
AD
6560@c Check if those paragraphs are still useful or not.
6561
6562@c @example
6563@c struct @{
6564@c int first_line, last_line;
6565@c int first_column, last_column;
6566@c @};
6567@c @end example
6568
6569@c Thus, to get the starting line number of the third component, you would
6570@c use @samp{@@3.first_line}.
bfa74976 6571
847bf1f5
AD
6572@c In order for the members of this structure to contain valid information,
6573@c you must make @code{yylex} supply this information about each token.
6574@c If you need only certain members, then @code{yylex} need only fill in
6575@c those members.
bfa74976 6576
847bf1f5 6577@c The use of this feature makes the parser noticeably slower.
18b519c0 6578@end deffn
847bf1f5 6579
18b519c0 6580@deffn {Value} @@@var{n}
847bf1f5 6581@findex @@@var{n}
95923bd6 6582Acts like a structure variable containing information on the textual location
847bf1f5
AD
6583of the @var{n}th component of the current rule. @xref{Locations, ,
6584Tracking Locations}.
18b519c0 6585@end deffn
bfa74976 6586
f7ab6a50
PE
6587@node Internationalization
6588@section Parser Internationalization
6589@cindex internationalization
6590@cindex i18n
6591@cindex NLS
6592@cindex gettext
6593@cindex bison-po
6594
6595A Bison-generated parser can print diagnostics, including error and
6596tracing messages. By default, they appear in English. However, Bison
f8e1c9e5
AD
6597also supports outputting diagnostics in the user's native language. To
6598make this work, the user should set the usual environment variables.
6599@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
6600For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
8a4281b9 6601set the user's locale to French Canadian using the UTF-8
f7ab6a50
PE
6602encoding. The exact set of available locales depends on the user's
6603installation.
6604
6605The maintainer of a package that uses a Bison-generated parser enables
6606the internationalization of the parser's output through the following
8a4281b9
JD
6607steps. Here we assume a package that uses GNU Autoconf and
6608GNU Automake.
f7ab6a50
PE
6609
6610@enumerate
6611@item
30757c8c 6612@cindex bison-i18n.m4
8a4281b9 6613Into the directory containing the GNU Autoconf macros used
f7ab6a50
PE
6614by the package---often called @file{m4}---copy the
6615@file{bison-i18n.m4} file installed by Bison under
6616@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
6617For example:
6618
6619@example
6620cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
6621@end example
6622
6623@item
30757c8c
PE
6624@findex BISON_I18N
6625@vindex BISON_LOCALEDIR
6626@vindex YYENABLE_NLS
f7ab6a50
PE
6627In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
6628invocation, add an invocation of @code{BISON_I18N}. This macro is
6629defined in the file @file{bison-i18n.m4} that you copied earlier. It
6630causes @samp{configure} to find the value of the
30757c8c
PE
6631@code{BISON_LOCALEDIR} variable, and it defines the source-language
6632symbol @code{YYENABLE_NLS} to enable translations in the
6633Bison-generated parser.
f7ab6a50
PE
6634
6635@item
6636In the @code{main} function of your program, designate the directory
6637containing Bison's runtime message catalog, through a call to
6638@samp{bindtextdomain} with domain name @samp{bison-runtime}.
6639For example:
6640
6641@example
6642bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
6643@end example
6644
6645Typically this appears after any other call @code{bindtextdomain
6646(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
6647@samp{BISON_LOCALEDIR} to be defined as a string through the
6648@file{Makefile}.
6649
6650@item
6651In the @file{Makefile.am} that controls the compilation of the @code{main}
6652function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
6653either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
6654
6655@example
6656DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6657@end example
6658
6659or:
6660
6661@example
6662AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6663@end example
6664
6665@item
6666Finally, invoke the command @command{autoreconf} to generate the build
6667infrastructure.
6668@end enumerate
6669
bfa74976 6670
342b8b6e 6671@node Algorithm
13863333
AD
6672@chapter The Bison Parser Algorithm
6673@cindex Bison parser algorithm
bfa74976
RS
6674@cindex algorithm of parser
6675@cindex shifting
6676@cindex reduction
6677@cindex parser stack
6678@cindex stack, parser
6679
6680As Bison reads tokens, it pushes them onto a stack along with their
6681semantic values. The stack is called the @dfn{parser stack}. Pushing a
6682token is traditionally called @dfn{shifting}.
6683
6684For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
6685@samp{3} to come. The stack will have four elements, one for each token
6686that was shifted.
6687
6688But the stack does not always have an element for each token read. When
6689the last @var{n} tokens and groupings shifted match the components of a
6690grammar rule, they can be combined according to that rule. This is called
6691@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
6692single grouping whose symbol is the result (left hand side) of that rule.
6693Running the rule's action is part of the process of reduction, because this
6694is what computes the semantic value of the resulting grouping.
6695
6696For example, if the infix calculator's parser stack contains this:
6697
6698@example
66991 + 5 * 3
6700@end example
6701
6702@noindent
6703and the next input token is a newline character, then the last three
6704elements can be reduced to 15 via the rule:
6705
6706@example
6707expr: expr '*' expr;
6708@end example
6709
6710@noindent
6711Then the stack contains just these three elements:
6712
6713@example
67141 + 15
6715@end example
6716
6717@noindent
6718At this point, another reduction can be made, resulting in the single value
671916. Then the newline token can be shifted.
6720
6721The parser tries, by shifts and reductions, to reduce the entire input down
6722to a single grouping whose symbol is the grammar's start-symbol
6723(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6724
6725This kind of parser is known in the literature as a bottom-up parser.
6726
6727@menu
742e4900 6728* Lookahead:: Parser looks one token ahead when deciding what to do.
bfa74976
RS
6729* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6730* Precedence:: Operator precedence works by resolving conflicts.
6731* Contextual Precedence:: When an operator's precedence depends on context.
6732* Parser States:: The parser is a finite-state-machine with stack.
6733* Reduce/Reduce:: When two rules are applicable in the same situation.
f5f419de 6734* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 6735* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 6736* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
6737@end menu
6738
742e4900
JD
6739@node Lookahead
6740@section Lookahead Tokens
6741@cindex lookahead token
bfa74976
RS
6742
6743The Bison parser does @emph{not} always reduce immediately as soon as the
6744last @var{n} tokens and groupings match a rule. This is because such a
6745simple strategy is inadequate to handle most languages. Instead, when a
6746reduction is possible, the parser sometimes ``looks ahead'' at the next
6747token in order to decide what to do.
6748
6749When a token is read, it is not immediately shifted; first it becomes the
742e4900 6750@dfn{lookahead token}, which is not on the stack. Now the parser can
bfa74976 6751perform one or more reductions of tokens and groupings on the stack, while
742e4900
JD
6752the lookahead token remains off to the side. When no more reductions
6753should take place, the lookahead token is shifted onto the stack. This
bfa74976 6754does not mean that all possible reductions have been done; depending on the
742e4900 6755token type of the lookahead token, some rules may choose to delay their
bfa74976
RS
6756application.
6757
742e4900 6758Here is a simple case where lookahead is needed. These three rules define
bfa74976
RS
6759expressions which contain binary addition operators and postfix unary
6760factorial operators (@samp{!}), and allow parentheses for grouping.
6761
6762@example
6763@group
6764expr: term '+' expr
6765 | term
6766 ;
6767@end group
6768
6769@group
6770term: '(' expr ')'
6771 | term '!'
6772 | NUMBER
6773 ;
6774@end group
6775@end example
6776
6777Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6778should be done? If the following token is @samp{)}, then the first three
6779tokens must be reduced to form an @code{expr}. This is the only valid
6780course, because shifting the @samp{)} would produce a sequence of symbols
6781@w{@code{term ')'}}, and no rule allows this.
6782
6783If the following token is @samp{!}, then it must be shifted immediately so
6784that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6785parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6786@code{expr}. It would then be impossible to shift the @samp{!} because
6787doing so would produce on the stack the sequence of symbols @code{expr
6788'!'}. No rule allows that sequence.
6789
6790@vindex yychar
32c29292
JD
6791@vindex yylval
6792@vindex yylloc
742e4900 6793The lookahead token is stored in the variable @code{yychar}.
32c29292
JD
6794Its semantic value and location, if any, are stored in the variables
6795@code{yylval} and @code{yylloc}.
bfa74976
RS
6796@xref{Action Features, ,Special Features for Use in Actions}.
6797
342b8b6e 6798@node Shift/Reduce
bfa74976
RS
6799@section Shift/Reduce Conflicts
6800@cindex conflicts
6801@cindex shift/reduce conflicts
6802@cindex dangling @code{else}
6803@cindex @code{else}, dangling
6804
6805Suppose we are parsing a language which has if-then and if-then-else
6806statements, with a pair of rules like this:
6807
6808@example
6809@group
6810if_stmt:
6811 IF expr THEN stmt
6812 | IF expr THEN stmt ELSE stmt
6813 ;
6814@end group
6815@end example
6816
6817@noindent
6818Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6819terminal symbols for specific keyword tokens.
6820
742e4900 6821When the @code{ELSE} token is read and becomes the lookahead token, the
bfa74976
RS
6822contents of the stack (assuming the input is valid) are just right for
6823reduction by the first rule. But it is also legitimate to shift the
6824@code{ELSE}, because that would lead to eventual reduction by the second
6825rule.
6826
6827This situation, where either a shift or a reduction would be valid, is
6828called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6829these conflicts by choosing to shift, unless otherwise directed by
6830operator precedence declarations. To see the reason for this, let's
6831contrast it with the other alternative.
6832
6833Since the parser prefers to shift the @code{ELSE}, the result is to attach
6834the else-clause to the innermost if-statement, making these two inputs
6835equivalent:
6836
6837@example
6838if x then if y then win (); else lose;
6839
6840if x then do; if y then win (); else lose; end;
6841@end example
6842
6843But if the parser chose to reduce when possible rather than shift, the
6844result would be to attach the else-clause to the outermost if-statement,
6845making these two inputs equivalent:
6846
6847@example
6848if x then if y then win (); else lose;
6849
6850if x then do; if y then win (); end; else lose;
6851@end example
6852
6853The conflict exists because the grammar as written is ambiguous: either
6854parsing of the simple nested if-statement is legitimate. The established
6855convention is that these ambiguities are resolved by attaching the
6856else-clause to the innermost if-statement; this is what Bison accomplishes
6857by choosing to shift rather than reduce. (It would ideally be cleaner to
6858write an unambiguous grammar, but that is very hard to do in this case.)
6859This particular ambiguity was first encountered in the specifications of
6860Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6861
6862To avoid warnings from Bison about predictable, legitimate shift/reduce
93d7dde9
JD
6863conflicts, use the @code{%expect @var{n}} declaration.
6864There will be no warning as long as the number of shift/reduce conflicts
6865is exactly @var{n}, and Bison will report an error if there is a
6866different number.
bfa74976
RS
6867@xref{Expect Decl, ,Suppressing Conflict Warnings}.
6868
6869The definition of @code{if_stmt} above is solely to blame for the
6870conflict, but the conflict does not actually appear without additional
ff7571c0
JD
6871rules. Here is a complete Bison grammar file that actually manifests
6872the conflict:
bfa74976
RS
6873
6874@example
6875@group
6876%token IF THEN ELSE variable
6877%%
6878@end group
6879@group
6880stmt: expr
6881 | if_stmt
6882 ;
6883@end group
6884
6885@group
6886if_stmt:
6887 IF expr THEN stmt
6888 | IF expr THEN stmt ELSE stmt
6889 ;
6890@end group
6891
6892expr: variable
6893 ;
6894@end example
6895
342b8b6e 6896@node Precedence
bfa74976
RS
6897@section Operator Precedence
6898@cindex operator precedence
6899@cindex precedence of operators
6900
6901Another situation where shift/reduce conflicts appear is in arithmetic
6902expressions. Here shifting is not always the preferred resolution; the
6903Bison declarations for operator precedence allow you to specify when to
6904shift and when to reduce.
6905
6906@menu
6907* Why Precedence:: An example showing why precedence is needed.
d78f0ac9
AD
6908* Using Precedence:: How to specify precedence and associativity.
6909* Precedence Only:: How to specify precedence only.
bfa74976
RS
6910* Precedence Examples:: How these features are used in the previous example.
6911* How Precedence:: How they work.
6912@end menu
6913
342b8b6e 6914@node Why Precedence
bfa74976
RS
6915@subsection When Precedence is Needed
6916
6917Consider the following ambiguous grammar fragment (ambiguous because the
6918input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6919
6920@example
6921@group
6922expr: expr '-' expr
6923 | expr '*' expr
6924 | expr '<' expr
6925 | '(' expr ')'
6926 @dots{}
6927 ;
6928@end group
6929@end example
6930
6931@noindent
6932Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
14ded682
AD
6933should it reduce them via the rule for the subtraction operator? It
6934depends on the next token. Of course, if the next token is @samp{)}, we
6935must reduce; shifting is invalid because no single rule can reduce the
6936token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6937the next token is @samp{*} or @samp{<}, we have a choice: either
6938shifting or reduction would allow the parse to complete, but with
6939different results.
6940
6941To decide which one Bison should do, we must consider the results. If
6942the next operator token @var{op} is shifted, then it must be reduced
6943first in order to permit another opportunity to reduce the difference.
6944The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6945hand, if the subtraction is reduced before shifting @var{op}, the result
6946is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6947reduce should depend on the relative precedence of the operators
6948@samp{-} and @var{op}: @samp{*} should be shifted first, but not
6949@samp{<}.
bfa74976
RS
6950
6951@cindex associativity
6952What about input such as @w{@samp{1 - 2 - 5}}; should this be
14ded682
AD
6953@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6954operators we prefer the former, which is called @dfn{left association}.
6955The latter alternative, @dfn{right association}, is desirable for
6956assignment operators. The choice of left or right association is a
6957matter of whether the parser chooses to shift or reduce when the stack
742e4900 6958contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
14ded682 6959makes right-associativity.
bfa74976 6960
342b8b6e 6961@node Using Precedence
bfa74976
RS
6962@subsection Specifying Operator Precedence
6963@findex %left
bfa74976 6964@findex %nonassoc
d78f0ac9
AD
6965@findex %precedence
6966@findex %right
bfa74976
RS
6967
6968Bison allows you to specify these choices with the operator precedence
6969declarations @code{%left} and @code{%right}. Each such declaration
6970contains a list of tokens, which are operators whose precedence and
6971associativity is being declared. The @code{%left} declaration makes all
6972those operators left-associative and the @code{%right} declaration makes
6973them right-associative. A third alternative is @code{%nonassoc}, which
6974declares that it is a syntax error to find the same operator twice ``in a
6975row''.
d78f0ac9
AD
6976The last alternative, @code{%precedence}, allows to define only
6977precedence and no associativity at all. As a result, any
6978associativity-related conflict that remains will be reported as an
6979compile-time error. The directive @code{%nonassoc} creates run-time
6980error: using the operator in a associative way is a syntax error. The
6981directive @code{%precedence} creates compile-time errors: an operator
6982@emph{can} be involved in an associativity-related conflict, contrary to
6983what expected the grammar author.
bfa74976
RS
6984
6985The relative precedence of different operators is controlled by the
d78f0ac9
AD
6986order in which they are declared. The first precedence/associativity
6987declaration in the file declares the operators whose
bfa74976
RS
6988precedence is lowest, the next such declaration declares the operators
6989whose precedence is a little higher, and so on.
6990
d78f0ac9
AD
6991@node Precedence Only
6992@subsection Specifying Precedence Only
6993@findex %precedence
6994
8a4281b9 6995Since POSIX Yacc defines only @code{%left}, @code{%right}, and
d78f0ac9
AD
6996@code{%nonassoc}, which all defines precedence and associativity, little
6997attention is paid to the fact that precedence cannot be defined without
6998defining associativity. Yet, sometimes, when trying to solve a
6999conflict, precedence suffices. In such a case, using @code{%left},
7000@code{%right}, or @code{%nonassoc} might hide future (associativity
7001related) conflicts that would remain hidden.
7002
7003The dangling @code{else} ambiguity (@pxref{Shift/Reduce, , Shift/Reduce
f50bfcd6 7004Conflicts}) can be solved explicitly. This shift/reduce conflicts occurs
d78f0ac9
AD
7005in the following situation, where the period denotes the current parsing
7006state:
7007
7008@example
7009if @var{e1} then if @var{e2} then @var{s1} . else @var{s2}
7010@end example
7011
7012The conflict involves the reduction of the rule @samp{IF expr THEN
7013stmt}, which precedence is by default that of its last token
7014(@code{THEN}), and the shifting of the token @code{ELSE}. The usual
7015disambiguation (attach the @code{else} to the closest @code{if}),
7016shifting must be preferred, i.e., the precedence of @code{ELSE} must be
7017higher than that of @code{THEN}. But neither is expected to be involved
7018in an associativity related conflict, which can be specified as follows.
7019
7020@example
7021%precedence THEN
7022%precedence ELSE
7023@end example
7024
7025The unary-minus is another typical example where associativity is
7026usually over-specified, see @ref{Infix Calc, , Infix Notation
f50bfcd6 7027Calculator: @code{calc}}. The @code{%left} directive is traditionally
d78f0ac9
AD
7028used to declare the precedence of @code{NEG}, which is more than needed
7029since it also defines its associativity. While this is harmless in the
7030traditional example, who knows how @code{NEG} might be used in future
7031evolutions of the grammar@dots{}
7032
342b8b6e 7033@node Precedence Examples
bfa74976
RS
7034@subsection Precedence Examples
7035
7036In our example, we would want the following declarations:
7037
7038@example
7039%left '<'
7040%left '-'
7041%left '*'
7042@end example
7043
7044In a more complete example, which supports other operators as well, we
7045would declare them in groups of equal precedence. For example, @code{'+'} is
7046declared with @code{'-'}:
7047
7048@example
7049%left '<' '>' '=' NE LE GE
7050%left '+' '-'
7051%left '*' '/'
7052@end example
7053
7054@noindent
7055(Here @code{NE} and so on stand for the operators for ``not equal''
7056and so on. We assume that these tokens are more than one character long
7057and therefore are represented by names, not character literals.)
7058
342b8b6e 7059@node How Precedence
bfa74976
RS
7060@subsection How Precedence Works
7061
7062The first effect of the precedence declarations is to assign precedence
7063levels to the terminal symbols declared. The second effect is to assign
704a47c4
AD
7064precedence levels to certain rules: each rule gets its precedence from
7065the last terminal symbol mentioned in the components. (You can also
7066specify explicitly the precedence of a rule. @xref{Contextual
7067Precedence, ,Context-Dependent Precedence}.)
7068
7069Finally, the resolution of conflicts works by comparing the precedence
742e4900 7070of the rule being considered with that of the lookahead token. If the
704a47c4
AD
7071token's precedence is higher, the choice is to shift. If the rule's
7072precedence is higher, the choice is to reduce. If they have equal
7073precedence, the choice is made based on the associativity of that
7074precedence level. The verbose output file made by @samp{-v}
7075(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
7076resolved.
bfa74976
RS
7077
7078Not all rules and not all tokens have precedence. If either the rule or
742e4900 7079the lookahead token has no precedence, then the default is to shift.
bfa74976 7080
342b8b6e 7081@node Contextual Precedence
bfa74976
RS
7082@section Context-Dependent Precedence
7083@cindex context-dependent precedence
7084@cindex unary operator precedence
7085@cindex precedence, context-dependent
7086@cindex precedence, unary operator
7087@findex %prec
7088
7089Often the precedence of an operator depends on the context. This sounds
7090outlandish at first, but it is really very common. For example, a minus
7091sign typically has a very high precedence as a unary operator, and a
7092somewhat lower precedence (lower than multiplication) as a binary operator.
7093
d78f0ac9
AD
7094The Bison precedence declarations
7095can only be used once for a given token; so a token has
bfa74976
RS
7096only one precedence declared in this way. For context-dependent
7097precedence, you need to use an additional mechanism: the @code{%prec}
e0c471a9 7098modifier for rules.
bfa74976
RS
7099
7100The @code{%prec} modifier declares the precedence of a particular rule by
7101specifying a terminal symbol whose precedence should be used for that rule.
7102It's not necessary for that symbol to appear otherwise in the rule. The
7103modifier's syntax is:
7104
7105@example
7106%prec @var{terminal-symbol}
7107@end example
7108
7109@noindent
7110and it is written after the components of the rule. Its effect is to
7111assign the rule the precedence of @var{terminal-symbol}, overriding
7112the precedence that would be deduced for it in the ordinary way. The
7113altered rule precedence then affects how conflicts involving that rule
7114are resolved (@pxref{Precedence, ,Operator Precedence}).
7115
7116Here is how @code{%prec} solves the problem of unary minus. First, declare
7117a precedence for a fictitious terminal symbol named @code{UMINUS}. There
7118are no tokens of this type, but the symbol serves to stand for its
7119precedence:
7120
7121@example
7122@dots{}
7123%left '+' '-'
7124%left '*'
7125%left UMINUS
7126@end example
7127
7128Now the precedence of @code{UMINUS} can be used in specific rules:
7129
7130@example
7131@group
7132exp: @dots{}
7133 | exp '-' exp
7134 @dots{}
7135 | '-' exp %prec UMINUS
7136@end group
7137@end example
7138
91d2c560 7139@ifset defaultprec
39a06c25
PE
7140If you forget to append @code{%prec UMINUS} to the rule for unary
7141minus, Bison silently assumes that minus has its usual precedence.
7142This kind of problem can be tricky to debug, since one typically
7143discovers the mistake only by testing the code.
7144
22fccf95 7145The @code{%no-default-prec;} declaration makes it easier to discover
39a06c25
PE
7146this kind of problem systematically. It causes rules that lack a
7147@code{%prec} modifier to have no precedence, even if the last terminal
7148symbol mentioned in their components has a declared precedence.
7149
22fccf95 7150If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
39a06c25
PE
7151for all rules that participate in precedence conflict resolution.
7152Then you will see any shift/reduce conflict until you tell Bison how
7153to resolve it, either by changing your grammar or by adding an
7154explicit precedence. This will probably add declarations to the
7155grammar, but it helps to protect against incorrect rule precedences.
7156
22fccf95
PE
7157The effect of @code{%no-default-prec;} can be reversed by giving
7158@code{%default-prec;}, which is the default.
91d2c560 7159@end ifset
39a06c25 7160
342b8b6e 7161@node Parser States
bfa74976
RS
7162@section Parser States
7163@cindex finite-state machine
7164@cindex parser state
7165@cindex state (of parser)
7166
7167The function @code{yyparse} is implemented using a finite-state machine.
7168The values pushed on the parser stack are not simply token type codes; they
7169represent the entire sequence of terminal and nonterminal symbols at or
7170near the top of the stack. The current state collects all the information
7171about previous input which is relevant to deciding what to do next.
7172
742e4900
JD
7173Each time a lookahead token is read, the current parser state together
7174with the type of lookahead token are looked up in a table. This table
7175entry can say, ``Shift the lookahead token.'' In this case, it also
bfa74976
RS
7176specifies the new parser state, which is pushed onto the top of the
7177parser stack. Or it can say, ``Reduce using rule number @var{n}.''
7178This means that a certain number of tokens or groupings are taken off
7179the top of the stack, and replaced by one grouping. In other words,
7180that number of states are popped from the stack, and one new state is
7181pushed.
7182
742e4900 7183There is one other alternative: the table can say that the lookahead token
bfa74976
RS
7184is erroneous in the current state. This causes error processing to begin
7185(@pxref{Error Recovery}).
7186
342b8b6e 7187@node Reduce/Reduce
bfa74976
RS
7188@section Reduce/Reduce Conflicts
7189@cindex reduce/reduce conflict
7190@cindex conflicts, reduce/reduce
7191
7192A reduce/reduce conflict occurs if there are two or more rules that apply
7193to the same sequence of input. This usually indicates a serious error
7194in the grammar.
7195
7196For example, here is an erroneous attempt to define a sequence
7197of zero or more @code{word} groupings.
7198
7199@example
7200sequence: /* empty */
7201 @{ printf ("empty sequence\n"); @}
7202 | maybeword
7203 | sequence word
7204 @{ printf ("added word %s\n", $2); @}
7205 ;
7206
7207maybeword: /* empty */
7208 @{ printf ("empty maybeword\n"); @}
7209 | word
7210 @{ printf ("single word %s\n", $1); @}
7211 ;
7212@end example
7213
7214@noindent
7215The error is an ambiguity: there is more than one way to parse a single
7216@code{word} into a @code{sequence}. It could be reduced to a
7217@code{maybeword} and then into a @code{sequence} via the second rule.
7218Alternatively, nothing-at-all could be reduced into a @code{sequence}
7219via the first rule, and this could be combined with the @code{word}
7220using the third rule for @code{sequence}.
7221
7222There is also more than one way to reduce nothing-at-all into a
7223@code{sequence}. This can be done directly via the first rule,
7224or indirectly via @code{maybeword} and then the second rule.
7225
7226You might think that this is a distinction without a difference, because it
7227does not change whether any particular input is valid or not. But it does
7228affect which actions are run. One parsing order runs the second rule's
7229action; the other runs the first rule's action and the third rule's action.
7230In this example, the output of the program changes.
7231
7232Bison resolves a reduce/reduce conflict by choosing to use the rule that
7233appears first in the grammar, but it is very risky to rely on this. Every
7234reduce/reduce conflict must be studied and usually eliminated. Here is the
7235proper way to define @code{sequence}:
7236
7237@example
7238sequence: /* empty */
7239 @{ printf ("empty sequence\n"); @}
7240 | sequence word
7241 @{ printf ("added word %s\n", $2); @}
7242 ;
7243@end example
7244
7245Here is another common error that yields a reduce/reduce conflict:
7246
7247@example
7248sequence: /* empty */
7249 | sequence words
7250 | sequence redirects
7251 ;
7252
7253words: /* empty */
7254 | words word
7255 ;
7256
7257redirects:/* empty */
7258 | redirects redirect
7259 ;
7260@end example
7261
7262@noindent
7263The intention here is to define a sequence which can contain either
7264@code{word} or @code{redirect} groupings. The individual definitions of
7265@code{sequence}, @code{words} and @code{redirects} are error-free, but the
7266three together make a subtle ambiguity: even an empty input can be parsed
7267in infinitely many ways!
7268
7269Consider: nothing-at-all could be a @code{words}. Or it could be two
7270@code{words} in a row, or three, or any number. It could equally well be a
7271@code{redirects}, or two, or any number. Or it could be a @code{words}
7272followed by three @code{redirects} and another @code{words}. And so on.
7273
7274Here are two ways to correct these rules. First, to make it a single level
7275of sequence:
7276
7277@example
7278sequence: /* empty */
7279 | sequence word
7280 | sequence redirect
7281 ;
7282@end example
7283
7284Second, to prevent either a @code{words} or a @code{redirects}
7285from being empty:
7286
7287@example
7288sequence: /* empty */
7289 | sequence words
7290 | sequence redirects
7291 ;
7292
7293words: word
7294 | words word
7295 ;
7296
7297redirects:redirect
7298 | redirects redirect
7299 ;
7300@end example
7301
342b8b6e 7302@node Mystery Conflicts
bfa74976
RS
7303@section Mysterious Reduce/Reduce Conflicts
7304
7305Sometimes reduce/reduce conflicts can occur that don't look warranted.
7306Here is an example:
7307
7308@example
7309@group
7310%token ID
7311
7312%%
7313def: param_spec return_spec ','
7314 ;
7315param_spec:
7316 type
7317 | name_list ':' type
7318 ;
7319@end group
7320@group
7321return_spec:
7322 type
7323 | name ':' type
7324 ;
7325@end group
7326@group
7327type: ID
7328 ;
7329@end group
7330@group
7331name: ID
7332 ;
7333name_list:
7334 name
7335 | name ',' name_list
7336 ;
7337@end group
7338@end example
7339
7340It would seem that this grammar can be parsed with only a single token
742e4900 7341of lookahead: when a @code{param_spec} is being read, an @code{ID} is
bfa74976 7342a @code{name} if a comma or colon follows, or a @code{type} if another
8a4281b9 7343@code{ID} follows. In other words, this grammar is LR(1).
bfa74976 7344
8a4281b9
JD
7345@cindex LR(1)
7346@cindex LALR(1)
eb45ef3b 7347However, for historical reasons, Bison cannot by default handle all
8a4281b9 7348LR(1) grammars.
eb45ef3b
JD
7349In this grammar, two contexts, that after an @code{ID} at the beginning
7350of a @code{param_spec} and likewise at the beginning of a
7351@code{return_spec}, are similar enough that Bison assumes they are the
7352same.
7353They appear similar because the same set of rules would be
bfa74976
RS
7354active---the rule for reducing to a @code{name} and that for reducing to
7355a @code{type}. Bison is unable to determine at that stage of processing
742e4900 7356that the rules would require different lookahead tokens in the two
bfa74976
RS
7357contexts, so it makes a single parser state for them both. Combining
7358the two contexts causes a conflict later. In parser terminology, this
8a4281b9 7359occurrence means that the grammar is not LALR(1).
bfa74976 7360
eb45ef3b 7361For many practical grammars (specifically those that fall into the
35c1e5f0
JD
7362non-LR(1) class), the limitations of LALR(1) result in difficulties
7363beyond just mysterious reduce/reduce conflicts. The best way to fix
7364all these problems is to select a different parser table generation
7365algorithm. Either IELR(1) or canonical LR(1) would suffice, but the
7366former is more efficient and easier to debug during development.
7367@xref{%define Summary,,lr.type}, for details. (Bison's IELR(1) and
7368canonical LR(1) implementations are experimental. More user feedback
7369will help to stabilize them.)
eb45ef3b 7370
8a4281b9 7371If you instead wish to work around LALR(1)'s limitations, you
eb45ef3b
JD
7372can often fix a mysterious conflict by identifying the two parser states
7373that are being confused, and adding something to make them look
7374distinct. In the above example, adding one rule to
bfa74976
RS
7375@code{return_spec} as follows makes the problem go away:
7376
7377@example
7378@group
7379%token BOGUS
7380@dots{}
7381%%
7382@dots{}
7383return_spec:
7384 type
7385 | name ':' type
7386 /* This rule is never used. */
7387 | ID BOGUS
7388 ;
7389@end group
7390@end example
7391
7392This corrects the problem because it introduces the possibility of an
7393additional active rule in the context after the @code{ID} at the beginning of
7394@code{return_spec}. This rule is not active in the corresponding context
7395in a @code{param_spec}, so the two contexts receive distinct parser states.
7396As long as the token @code{BOGUS} is never generated by @code{yylex},
7397the added rule cannot alter the way actual input is parsed.
7398
7399In this particular example, there is another way to solve the problem:
7400rewrite the rule for @code{return_spec} to use @code{ID} directly
7401instead of via @code{name}. This also causes the two confusing
7402contexts to have different sets of active rules, because the one for
7403@code{return_spec} activates the altered rule for @code{return_spec}
7404rather than the one for @code{name}.
7405
7406@example
7407param_spec:
7408 type
7409 | name_list ':' type
7410 ;
7411return_spec:
7412 type
7413 | ID ':' type
7414 ;
7415@end example
7416
8a4281b9 7417For a more detailed exposition of LALR(1) parsers and parser
5e528941 7418generators, @pxref{Bibliography,,DeRemer 1982}.
e054b190 7419
fae437e8 7420@node Generalized LR Parsing
8a4281b9
JD
7421@section Generalized LR (GLR) Parsing
7422@cindex GLR parsing
7423@cindex generalized LR (GLR) parsing
676385e2 7424@cindex ambiguous grammars
9d9b8b70 7425@cindex nondeterministic parsing
676385e2 7426
fae437e8
AD
7427Bison produces @emph{deterministic} parsers that choose uniquely
7428when to reduce and which reduction to apply
742e4900 7429based on a summary of the preceding input and on one extra token of lookahead.
676385e2
PH
7430As a result, normal Bison handles a proper subset of the family of
7431context-free languages.
fae437e8 7432Ambiguous grammars, since they have strings with more than one possible
676385e2
PH
7433sequence of reductions cannot have deterministic parsers in this sense.
7434The same is true of languages that require more than one symbol of
742e4900 7435lookahead, since the parser lacks the information necessary to make a
676385e2 7436decision at the point it must be made in a shift-reduce parser.
fae437e8 7437Finally, as previously mentioned (@pxref{Mystery Conflicts}),
eb45ef3b 7438there are languages where Bison's default choice of how to
676385e2
PH
7439summarize the input seen so far loses necessary information.
7440
7441When you use the @samp{%glr-parser} declaration in your grammar file,
7442Bison generates a parser that uses a different algorithm, called
8a4281b9 7443Generalized LR (or GLR). A Bison GLR
c827f760 7444parser uses the same basic
676385e2
PH
7445algorithm for parsing as an ordinary Bison parser, but behaves
7446differently in cases where there is a shift-reduce conflict that has not
fae437e8 7447been resolved by precedence rules (@pxref{Precedence}) or a
8a4281b9 7448reduce-reduce conflict. When a GLR parser encounters such a
c827f760 7449situation, it
fae437e8 7450effectively @emph{splits} into a several parsers, one for each possible
676385e2
PH
7451shift or reduction. These parsers then proceed as usual, consuming
7452tokens in lock-step. Some of the stacks may encounter other conflicts
fae437e8 7453and split further, with the result that instead of a sequence of states,
8a4281b9 7454a Bison GLR parsing stack is what is in effect a tree of states.
676385e2
PH
7455
7456In effect, each stack represents a guess as to what the proper parse
7457is. Additional input may indicate that a guess was wrong, in which case
7458the appropriate stack silently disappears. Otherwise, the semantics
fae437e8 7459actions generated in each stack are saved, rather than being executed
676385e2 7460immediately. When a stack disappears, its saved semantic actions never
fae437e8 7461get executed. When a reduction causes two stacks to become equivalent,
676385e2
PH
7462their sets of semantic actions are both saved with the state that
7463results from the reduction. We say that two stacks are equivalent
fae437e8 7464when they both represent the same sequence of states,
676385e2
PH
7465and each pair of corresponding states represents a
7466grammar symbol that produces the same segment of the input token
7467stream.
7468
7469Whenever the parser makes a transition from having multiple
eb45ef3b 7470states to having one, it reverts to the normal deterministic parsing
676385e2
PH
7471algorithm, after resolving and executing the saved-up actions.
7472At this transition, some of the states on the stack will have semantic
7473values that are sets (actually multisets) of possible actions. The
7474parser tries to pick one of the actions by first finding one whose rule
7475has the highest dynamic precedence, as set by the @samp{%dprec}
fae437e8 7476declaration. Otherwise, if the alternative actions are not ordered by
676385e2 7477precedence, but there the same merging function is declared for both
fae437e8 7478rules by the @samp{%merge} declaration,
676385e2
PH
7479Bison resolves and evaluates both and then calls the merge function on
7480the result. Otherwise, it reports an ambiguity.
7481
8a4281b9
JD
7482It is possible to use a data structure for the GLR parsing tree that
7483permits the processing of any LR(1) grammar in linear time (in the
c827f760 7484size of the input), any unambiguous (not necessarily
8a4281b9 7485LR(1)) grammar in
fae437e8 7486quadratic worst-case time, and any general (possibly ambiguous)
676385e2
PH
7487context-free grammar in cubic worst-case time. However, Bison currently
7488uses a simpler data structure that requires time proportional to the
7489length of the input times the maximum number of stacks required for any
9d9b8b70 7490prefix of the input. Thus, really ambiguous or nondeterministic
676385e2
PH
7491grammars can require exponential time and space to process. Such badly
7492behaving examples, however, are not generally of practical interest.
9d9b8b70 7493Usually, nondeterminism in a grammar is local---the parser is ``in
676385e2 7494doubt'' only for a few tokens at a time. Therefore, the current data
8a4281b9 7495structure should generally be adequate. On LR(1) portions of a
eb45ef3b 7496grammar, in particular, it is only slightly slower than with the
8a4281b9 7497deterministic LR(1) Bison parser.
676385e2 7498
5e528941
JD
7499For a more detailed exposition of GLR parsers, @pxref{Bibliography,,Scott
75002000}.
f6481e2f 7501
1a059451
PE
7502@node Memory Management
7503@section Memory Management, and How to Avoid Memory Exhaustion
7504@cindex memory exhaustion
7505@cindex memory management
bfa74976
RS
7506@cindex stack overflow
7507@cindex parser stack overflow
7508@cindex overflow of parser stack
7509
1a059451 7510The Bison parser stack can run out of memory if too many tokens are shifted and
bfa74976 7511not reduced. When this happens, the parser function @code{yyparse}
1a059451 7512calls @code{yyerror} and then returns 2.
bfa74976 7513
c827f760 7514Because Bison parsers have growing stacks, hitting the upper limit
d1a1114f
AD
7515usually results from using a right recursion instead of a left
7516recursion, @xref{Recursion, ,Recursive Rules}.
7517
bfa74976
RS
7518@vindex YYMAXDEPTH
7519By defining the macro @code{YYMAXDEPTH}, you can control how deep the
1a059451 7520parser stack can become before memory is exhausted. Define the
bfa74976
RS
7521macro with a value that is an integer. This value is the maximum number
7522of tokens that can be shifted (and not reduced) before overflow.
bfa74976
RS
7523
7524The stack space allowed is not necessarily allocated. If you specify a
1a059451 7525large value for @code{YYMAXDEPTH}, the parser normally allocates a small
bfa74976
RS
7526stack at first, and then makes it bigger by stages as needed. This
7527increasing allocation happens automatically and silently. Therefore,
7528you do not need to make @code{YYMAXDEPTH} painfully small merely to save
7529space for ordinary inputs that do not need much stack.
7530
d7e14fc0
PE
7531However, do not allow @code{YYMAXDEPTH} to be a value so large that
7532arithmetic overflow could occur when calculating the size of the stack
7533space. Also, do not allow @code{YYMAXDEPTH} to be less than
7534@code{YYINITDEPTH}.
7535
bfa74976
RS
7536@cindex default stack limit
7537The default value of @code{YYMAXDEPTH}, if you do not define it, is
753810000.
7539
7540@vindex YYINITDEPTH
7541You can control how much stack is allocated initially by defining the
eb45ef3b
JD
7542macro @code{YYINITDEPTH} to a positive integer. For the deterministic
7543parser in C, this value must be a compile-time constant
d7e14fc0
PE
7544unless you are assuming C99 or some other target language or compiler
7545that allows variable-length arrays. The default is 200.
7546
1a059451 7547Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
bfa74976 7548
20be2f92
PH
7549You can generate a deterministic parser containing C++ user code from
7550the default (C) skeleton, as well as from the C++ skeleton
7551(@pxref{C++ Parsers}). However, if you do use the default skeleton
7552and want to allow the parsing stack to grow,
7553be careful not to use semantic types or location types that require
7554non-trivial copy constructors.
7555The C skeleton bypasses these constructors when copying data to
7556new, larger stacks.
d1a1114f 7557
342b8b6e 7558@node Error Recovery
bfa74976
RS
7559@chapter Error Recovery
7560@cindex error recovery
7561@cindex recovery from errors
7562
6e649e65 7563It is not usually acceptable to have a program terminate on a syntax
bfa74976
RS
7564error. For example, a compiler should recover sufficiently to parse the
7565rest of the input file and check it for errors; a calculator should accept
7566another expression.
7567
7568In a simple interactive command parser where each input is one line, it may
7569be sufficient to allow @code{yyparse} to return 1 on error and have the
7570caller ignore the rest of the input line when that happens (and then call
7571@code{yyparse} again). But this is inadequate for a compiler, because it
7572forgets all the syntactic context leading up to the error. A syntax error
7573deep within a function in the compiler input should not cause the compiler
7574to treat the following line like the beginning of a source file.
7575
7576@findex error
7577You can define how to recover from a syntax error by writing rules to
7578recognize the special token @code{error}. This is a terminal symbol that
7579is always defined (you need not declare it) and reserved for error
7580handling. The Bison parser generates an @code{error} token whenever a
7581syntax error happens; if you have provided a rule to recognize this token
13863333 7582in the current context, the parse can continue.
bfa74976
RS
7583
7584For example:
7585
7586@example
7587stmnts: /* empty string */
7588 | stmnts '\n'
7589 | stmnts exp '\n'
7590 | stmnts error '\n'
7591@end example
7592
7593The fourth rule in this example says that an error followed by a newline
7594makes a valid addition to any @code{stmnts}.
7595
7596What happens if a syntax error occurs in the middle of an @code{exp}? The
7597error recovery rule, interpreted strictly, applies to the precise sequence
7598of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
7599the middle of an @code{exp}, there will probably be some additional tokens
7600and subexpressions on the stack after the last @code{stmnts}, and there
7601will be tokens to read before the next newline. So the rule is not
7602applicable in the ordinary way.
7603
7604But Bison can force the situation to fit the rule, by discarding part of
72f889cc
AD
7605the semantic context and part of the input. First it discards states
7606and objects from the stack until it gets back to a state in which the
bfa74976 7607@code{error} token is acceptable. (This means that the subexpressions
72f889cc
AD
7608already parsed are discarded, back to the last complete @code{stmnts}.)
7609At this point the @code{error} token can be shifted. Then, if the old
742e4900 7610lookahead token is not acceptable to be shifted next, the parser reads
bfa74976 7611tokens and discards them until it finds a token which is acceptable. In
72f889cc
AD
7612this example, Bison reads and discards input until the next newline so
7613that the fourth rule can apply. Note that discarded symbols are
7614possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
7615Discarded Symbols}, for a means to reclaim this memory.
bfa74976
RS
7616
7617The choice of error rules in the grammar is a choice of strategies for
7618error recovery. A simple and useful strategy is simply to skip the rest of
7619the current input line or current statement if an error is detected:
7620
7621@example
72d2299c 7622stmnt: error ';' /* On error, skip until ';' is read. */
bfa74976
RS
7623@end example
7624
7625It is also useful to recover to the matching close-delimiter of an
7626opening-delimiter that has already been parsed. Otherwise the
7627close-delimiter will probably appear to be unmatched, and generate another,
7628spurious error message:
7629
7630@example
7631primary: '(' expr ')'
7632 | '(' error ')'
7633 @dots{}
7634 ;
7635@end example
7636
7637Error recovery strategies are necessarily guesses. When they guess wrong,
7638one syntax error often leads to another. In the above example, the error
7639recovery rule guesses that an error is due to bad input within one
7640@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
7641middle of a valid @code{stmnt}. After the error recovery rule recovers
7642from the first error, another syntax error will be found straightaway,
7643since the text following the spurious semicolon is also an invalid
7644@code{stmnt}.
7645
7646To prevent an outpouring of error messages, the parser will output no error
7647message for another syntax error that happens shortly after the first; only
7648after three consecutive input tokens have been successfully shifted will
7649error messages resume.
7650
7651Note that rules which accept the @code{error} token may have actions, just
7652as any other rules can.
7653
7654@findex yyerrok
7655You can make error messages resume immediately by using the macro
7656@code{yyerrok} in an action. If you do this in the error rule's action, no
7657error messages will be suppressed. This macro requires no arguments;
7658@samp{yyerrok;} is a valid C statement.
7659
7660@findex yyclearin
742e4900 7661The previous lookahead token is reanalyzed immediately after an error. If
bfa74976
RS
7662this is unacceptable, then the macro @code{yyclearin} may be used to clear
7663this token. Write the statement @samp{yyclearin;} in the error rule's
7664action.
32c29292 7665@xref{Action Features, ,Special Features for Use in Actions}.
bfa74976 7666
6e649e65 7667For example, suppose that on a syntax error, an error handling routine is
bfa74976
RS
7668called that advances the input stream to some point where parsing should
7669once again commence. The next symbol returned by the lexical scanner is
742e4900 7670probably correct. The previous lookahead token ought to be discarded
bfa74976
RS
7671with @samp{yyclearin;}.
7672
7673@vindex YYRECOVERING
02103984
PE
7674The expression @code{YYRECOVERING ()} yields 1 when the parser
7675is recovering from a syntax error, and 0 otherwise.
7676Syntax error diagnostics are suppressed while recovering from a syntax
7677error.
bfa74976 7678
342b8b6e 7679@node Context Dependency
bfa74976
RS
7680@chapter Handling Context Dependencies
7681
7682The Bison paradigm is to parse tokens first, then group them into larger
7683syntactic units. In many languages, the meaning of a token is affected by
7684its context. Although this violates the Bison paradigm, certain techniques
7685(known as @dfn{kludges}) may enable you to write Bison parsers for such
7686languages.
7687
7688@menu
7689* Semantic Tokens:: Token parsing can depend on the semantic context.
7690* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
7691* Tie-in Recovery:: Lexical tie-ins have implications for how
7692 error recovery rules must be written.
7693@end menu
7694
7695(Actually, ``kludge'' means any technique that gets its job done but is
7696neither clean nor robust.)
7697
342b8b6e 7698@node Semantic Tokens
bfa74976
RS
7699@section Semantic Info in Token Types
7700
7701The C language has a context dependency: the way an identifier is used
7702depends on what its current meaning is. For example, consider this:
7703
7704@example
7705foo (x);
7706@end example
7707
7708This looks like a function call statement, but if @code{foo} is a typedef
7709name, then this is actually a declaration of @code{x}. How can a Bison
7710parser for C decide how to parse this input?
7711
8a4281b9 7712The method used in GNU C is to have two different token types,
bfa74976
RS
7713@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
7714identifier, it looks up the current declaration of the identifier in order
7715to decide which token type to return: @code{TYPENAME} if the identifier is
7716declared as a typedef, @code{IDENTIFIER} otherwise.
7717
7718The grammar rules can then express the context dependency by the choice of
7719token type to recognize. @code{IDENTIFIER} is accepted as an expression,
7720but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
7721@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
7722is @emph{not} significant, such as in declarations that can shadow a
7723typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
7724accepted---there is one rule for each of the two token types.
7725
7726This technique is simple to use if the decision of which kinds of
7727identifiers to allow is made at a place close to where the identifier is
7728parsed. But in C this is not always so: C allows a declaration to
7729redeclare a typedef name provided an explicit type has been specified
7730earlier:
7731
7732@example
3a4f411f
PE
7733typedef int foo, bar;
7734int baz (void)
7735@{
7736 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
7737 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
7738 return foo (bar);
7739@}
bfa74976
RS
7740@end example
7741
7742Unfortunately, the name being declared is separated from the declaration
7743construct itself by a complicated syntactic structure---the ``declarator''.
7744
9ecbd125 7745As a result, part of the Bison parser for C needs to be duplicated, with
14ded682
AD
7746all the nonterminal names changed: once for parsing a declaration in
7747which a typedef name can be redefined, and once for parsing a
7748declaration in which that can't be done. Here is a part of the
7749duplication, with actions omitted for brevity:
bfa74976
RS
7750
7751@example
7752initdcl:
7753 declarator maybeasm '='
7754 init
7755 | declarator maybeasm
7756 ;
7757
7758notype_initdcl:
7759 notype_declarator maybeasm '='
7760 init
7761 | notype_declarator maybeasm
7762 ;
7763@end example
7764
7765@noindent
7766Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
7767cannot. The distinction between @code{declarator} and
7768@code{notype_declarator} is the same sort of thing.
7769
7770There is some similarity between this technique and a lexical tie-in
7771(described next), in that information which alters the lexical analysis is
7772changed during parsing by other parts of the program. The difference is
7773here the information is global, and is used for other purposes in the
7774program. A true lexical tie-in has a special-purpose flag controlled by
7775the syntactic context.
7776
342b8b6e 7777@node Lexical Tie-ins
bfa74976
RS
7778@section Lexical Tie-ins
7779@cindex lexical tie-in
7780
7781One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7782which is set by Bison actions, whose purpose is to alter the way tokens are
7783parsed.
7784
7785For example, suppose we have a language vaguely like C, but with a special
7786construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7787an expression in parentheses in which all integers are hexadecimal. In
7788particular, the token @samp{a1b} must be treated as an integer rather than
7789as an identifier if it appears in that context. Here is how you can do it:
7790
7791@example
7792@group
7793%@{
38a92d50
PE
7794 int hexflag;
7795 int yylex (void);
7796 void yyerror (char const *);
bfa74976
RS
7797%@}
7798%%
7799@dots{}
7800@end group
7801@group
7802expr: IDENTIFIER
7803 | constant
7804 | HEX '('
7805 @{ hexflag = 1; @}
7806 expr ')'
7807 @{ hexflag = 0;
7808 $$ = $4; @}
7809 | expr '+' expr
7810 @{ $$ = make_sum ($1, $3); @}
7811 @dots{}
7812 ;
7813@end group
7814
7815@group
7816constant:
7817 INTEGER
7818 | STRING
7819 ;
7820@end group
7821@end example
7822
7823@noindent
7824Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
7825it is nonzero, all integers are parsed in hexadecimal, and tokens starting
7826with letters are parsed as integers if possible.
7827
ff7571c0
JD
7828The declaration of @code{hexflag} shown in the prologue of the grammar
7829file is needed to make it accessible to the actions (@pxref{Prologue,
7830,The Prologue}). You must also write the code in @code{yylex} to obey
7831the flag.
bfa74976 7832
342b8b6e 7833@node Tie-in Recovery
bfa74976
RS
7834@section Lexical Tie-ins and Error Recovery
7835
7836Lexical tie-ins make strict demands on any error recovery rules you have.
7837@xref{Error Recovery}.
7838
7839The reason for this is that the purpose of an error recovery rule is to
7840abort the parsing of one construct and resume in some larger construct.
7841For example, in C-like languages, a typical error recovery rule is to skip
7842tokens until the next semicolon, and then start a new statement, like this:
7843
7844@example
7845stmt: expr ';'
7846 | IF '(' expr ')' stmt @{ @dots{} @}
7847 @dots{}
7848 error ';'
7849 @{ hexflag = 0; @}
7850 ;
7851@end example
7852
7853If there is a syntax error in the middle of a @samp{hex (@var{expr})}
7854construct, this error rule will apply, and then the action for the
7855completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
7856remain set for the entire rest of the input, or until the next @code{hex}
7857keyword, causing identifiers to be misinterpreted as integers.
7858
7859To avoid this problem the error recovery rule itself clears @code{hexflag}.
7860
7861There may also be an error recovery rule that works within expressions.
7862For example, there could be a rule which applies within parentheses
7863and skips to the close-parenthesis:
7864
7865@example
7866@group
7867expr: @dots{}
7868 | '(' expr ')'
7869 @{ $$ = $2; @}
7870 | '(' error ')'
7871 @dots{}
7872@end group
7873@end example
7874
7875If this rule acts within the @code{hex} construct, it is not going to abort
7876that construct (since it applies to an inner level of parentheses within
7877the construct). Therefore, it should not clear the flag: the rest of
7878the @code{hex} construct should be parsed with the flag still in effect.
7879
7880What if there is an error recovery rule which might abort out of the
7881@code{hex} construct or might not, depending on circumstances? There is no
7882way you can write the action to determine whether a @code{hex} construct is
7883being aborted or not. So if you are using a lexical tie-in, you had better
7884make sure your error recovery rules are not of this kind. Each rule must
7885be such that you can be sure that it always will, or always won't, have to
7886clear the flag.
7887
ec3bc396
AD
7888@c ================================================== Debugging Your Parser
7889
342b8b6e 7890@node Debugging
bfa74976 7891@chapter Debugging Your Parser
ec3bc396
AD
7892
7893Developing a parser can be a challenge, especially if you don't
7894understand the algorithm (@pxref{Algorithm, ,The Bison Parser
7895Algorithm}). Even so, sometimes a detailed description of the automaton
7896can help (@pxref{Understanding, , Understanding Your Parser}), or
7897tracing the execution of the parser can give some insight on why it
7898behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
7899
7900@menu
7901* Understanding:: Understanding the structure of your parser.
7902* Tracing:: Tracing the execution of your parser.
7903@end menu
7904
7905@node Understanding
7906@section Understanding Your Parser
7907
7908As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
7909Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
7910frequent than one would hope), looking at this automaton is required to
7911tune or simply fix a parser. Bison provides two different
35fe0834 7912representation of it, either textually or graphically (as a DOT file).
ec3bc396
AD
7913
7914The textual file is generated when the options @option{--report} or
7915@option{--verbose} are specified, see @xref{Invocation, , Invoking
7916Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
ff7571c0
JD
7917the parser implementation file name, and adding @samp{.output}
7918instead. Therefore, if the grammar file is @file{foo.y}, then the
7919parser implementation file is called @file{foo.tab.c} by default. As
7920a consequence, the verbose output file is called @file{foo.output}.
ec3bc396
AD
7921
7922The following grammar file, @file{calc.y}, will be used in the sequel:
7923
7924@example
7925%token NUM STR
7926%left '+' '-'
7927%left '*'
7928%%
7929exp: exp '+' exp
7930 | exp '-' exp
7931 | exp '*' exp
7932 | exp '/' exp
7933 | NUM
7934 ;
7935useless: STR;
7936%%
7937@end example
7938
88bce5a2
AD
7939@command{bison} reports:
7940
7941@example
8f0d265e
JD
7942calc.y: warning: 1 nonterminal useless in grammar
7943calc.y: warning: 1 rule useless in grammar
cff03fb2
JD
7944calc.y:11.1-7: warning: nonterminal useless in grammar: useless
7945calc.y:11.10-12: warning: rule useless in grammar: useless: STR
5a99098d 7946calc.y: conflicts: 7 shift/reduce
88bce5a2
AD
7947@end example
7948
7949When given @option{--report=state}, in addition to @file{calc.tab.c}, it
7950creates a file @file{calc.output} with contents detailed below. The
7951order of the output and the exact presentation might vary, but the
7952interpretation is the same.
ec3bc396
AD
7953
7954The first section includes details on conflicts that were solved thanks
7955to precedence and/or associativity:
7956
7957@example
7958Conflict in state 8 between rule 2 and token '+' resolved as reduce.
7959Conflict in state 8 between rule 2 and token '-' resolved as reduce.
7960Conflict in state 8 between rule 2 and token '*' resolved as shift.
7961@exdent @dots{}
7962@end example
7963
7964@noindent
7965The next section lists states that still have conflicts.
7966
7967@example
5a99098d
PE
7968State 8 conflicts: 1 shift/reduce
7969State 9 conflicts: 1 shift/reduce
7970State 10 conflicts: 1 shift/reduce
7971State 11 conflicts: 4 shift/reduce
ec3bc396
AD
7972@end example
7973
7974@noindent
7975@cindex token, useless
7976@cindex useless token
7977@cindex nonterminal, useless
7978@cindex useless nonterminal
7979@cindex rule, useless
7980@cindex useless rule
7981The next section reports useless tokens, nonterminal and rules. Useless
7982nonterminals and rules are removed in order to produce a smaller parser,
7983but useless tokens are preserved, since they might be used by the
d80fb37a 7984scanner (note the difference between ``useless'' and ``unused''
ec3bc396
AD
7985below):
7986
7987@example
d80fb37a 7988Nonterminals useless in grammar:
ec3bc396
AD
7989 useless
7990
d80fb37a 7991Terminals unused in grammar:
ec3bc396
AD
7992 STR
7993
cff03fb2 7994Rules useless in grammar:
ec3bc396
AD
7995#6 useless: STR;
7996@end example
7997
7998@noindent
7999The next section reproduces the exact grammar that Bison used:
8000
8001@example
8002Grammar
8003
8004 Number, Line, Rule
88bce5a2 8005 0 5 $accept -> exp $end
ec3bc396
AD
8006 1 5 exp -> exp '+' exp
8007 2 6 exp -> exp '-' exp
8008 3 7 exp -> exp '*' exp
8009 4 8 exp -> exp '/' exp
8010 5 9 exp -> NUM
8011@end example
8012
8013@noindent
8014and reports the uses of the symbols:
8015
8016@example
8017Terminals, with rules where they appear
8018
88bce5a2 8019$end (0) 0
ec3bc396
AD
8020'*' (42) 3
8021'+' (43) 1
8022'-' (45) 2
8023'/' (47) 4
8024error (256)
8025NUM (258) 5
8026
8027Nonterminals, with rules where they appear
8028
88bce5a2 8029$accept (8)
ec3bc396
AD
8030 on left: 0
8031exp (9)
8032 on left: 1 2 3 4 5, on right: 0 1 2 3 4
8033@end example
8034
8035@noindent
8036@cindex item
8037@cindex pointed rule
8038@cindex rule, pointed
8039Bison then proceeds onto the automaton itself, describing each state
8040with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
8041item is a production rule together with a point (marked by @samp{.})
8042that the input cursor.
8043
8044@example
8045state 0
8046
88bce5a2 8047 $accept -> . exp $ (rule 0)
ec3bc396 8048
2a8d363a 8049 NUM shift, and go to state 1
ec3bc396 8050
2a8d363a 8051 exp go to state 2
ec3bc396
AD
8052@end example
8053
8054This reads as follows: ``state 0 corresponds to being at the very
8055beginning of the parsing, in the initial rule, right before the start
8056symbol (here, @code{exp}). When the parser returns to this state right
8057after having reduced a rule that produced an @code{exp}, the control
8058flow jumps to state 2. If there is no such transition on a nonterminal
742e4900 8059symbol, and the lookahead is a @code{NUM}, then this token is shifted on
ec3bc396 8060the parse stack, and the control flow jumps to state 1. Any other
742e4900 8061lookahead triggers a syntax error.''
ec3bc396
AD
8062
8063@cindex core, item set
8064@cindex item set core
8065@cindex kernel, item set
8066@cindex item set core
8067Even though the only active rule in state 0 seems to be rule 0, the
742e4900 8068report lists @code{NUM} as a lookahead token because @code{NUM} can be
ec3bc396
AD
8069at the beginning of any rule deriving an @code{exp}. By default Bison
8070reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
8071you want to see more detail you can invoke @command{bison} with
8072@option{--report=itemset} to list all the items, include those that can
8073be derived:
8074
8075@example
8076state 0
8077
88bce5a2 8078 $accept -> . exp $ (rule 0)
ec3bc396
AD
8079 exp -> . exp '+' exp (rule 1)
8080 exp -> . exp '-' exp (rule 2)
8081 exp -> . exp '*' exp (rule 3)
8082 exp -> . exp '/' exp (rule 4)
8083 exp -> . NUM (rule 5)
8084
8085 NUM shift, and go to state 1
8086
8087 exp go to state 2
8088@end example
8089
8090@noindent
8091In the state 1...
8092
8093@example
8094state 1
8095
8096 exp -> NUM . (rule 5)
8097
2a8d363a 8098 $default reduce using rule 5 (exp)
ec3bc396
AD
8099@end example
8100
8101@noindent
742e4900 8102the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
ec3bc396
AD
8103(@samp{$default}), the parser will reduce it. If it was coming from
8104state 0, then, after this reduction it will return to state 0, and will
8105jump to state 2 (@samp{exp: go to state 2}).
8106
8107@example
8108state 2
8109
88bce5a2 8110 $accept -> exp . $ (rule 0)
ec3bc396
AD
8111 exp -> exp . '+' exp (rule 1)
8112 exp -> exp . '-' exp (rule 2)
8113 exp -> exp . '*' exp (rule 3)
8114 exp -> exp . '/' exp (rule 4)
8115
2a8d363a
AD
8116 $ shift, and go to state 3
8117 '+' shift, and go to state 4
8118 '-' shift, and go to state 5
8119 '*' shift, and go to state 6
8120 '/' shift, and go to state 7
ec3bc396
AD
8121@end example
8122
8123@noindent
8124In state 2, the automaton can only shift a symbol. For instance,
742e4900 8125because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
ec3bc396
AD
8126@samp{+}, it will be shifted on the parse stack, and the automaton
8127control will jump to state 4, corresponding to the item @samp{exp -> exp
8128'+' . exp}. Since there is no default action, any other token than
6e649e65 8129those listed above will trigger a syntax error.
ec3bc396 8130
eb45ef3b 8131@cindex accepting state
ec3bc396
AD
8132The state 3 is named the @dfn{final state}, or the @dfn{accepting
8133state}:
8134
8135@example
8136state 3
8137
88bce5a2 8138 $accept -> exp $ . (rule 0)
ec3bc396 8139
2a8d363a 8140 $default accept
ec3bc396
AD
8141@end example
8142
8143@noindent
8144the initial rule is completed (the start symbol and the end
8145of input were read), the parsing exits successfully.
8146
8147The interpretation of states 4 to 7 is straightforward, and is left to
8148the reader.
8149
8150@example
8151state 4
8152
8153 exp -> exp '+' . exp (rule 1)
8154
2a8d363a 8155 NUM shift, and go to state 1
ec3bc396 8156
2a8d363a 8157 exp go to state 8
ec3bc396
AD
8158
8159state 5
8160
8161 exp -> exp '-' . exp (rule 2)
8162
2a8d363a 8163 NUM shift, and go to state 1
ec3bc396 8164
2a8d363a 8165 exp go to state 9
ec3bc396
AD
8166
8167state 6
8168
8169 exp -> exp '*' . exp (rule 3)
8170
2a8d363a 8171 NUM shift, and go to state 1
ec3bc396 8172
2a8d363a 8173 exp go to state 10
ec3bc396
AD
8174
8175state 7
8176
8177 exp -> exp '/' . exp (rule 4)
8178
2a8d363a 8179 NUM shift, and go to state 1
ec3bc396 8180
2a8d363a 8181 exp go to state 11
ec3bc396
AD
8182@end example
8183
5a99098d
PE
8184As was announced in beginning of the report, @samp{State 8 conflicts:
81851 shift/reduce}:
ec3bc396
AD
8186
8187@example
8188state 8
8189
8190 exp -> exp . '+' exp (rule 1)
8191 exp -> exp '+' exp . (rule 1)
8192 exp -> exp . '-' exp (rule 2)
8193 exp -> exp . '*' exp (rule 3)
8194 exp -> exp . '/' exp (rule 4)
8195
2a8d363a
AD
8196 '*' shift, and go to state 6
8197 '/' shift, and go to state 7
ec3bc396 8198
2a8d363a
AD
8199 '/' [reduce using rule 1 (exp)]
8200 $default reduce using rule 1 (exp)
ec3bc396
AD
8201@end example
8202
742e4900 8203Indeed, there are two actions associated to the lookahead @samp{/}:
ec3bc396
AD
8204either shifting (and going to state 7), or reducing rule 1. The
8205conflict means that either the grammar is ambiguous, or the parser lacks
8206information to make the right decision. Indeed the grammar is
8207ambiguous, as, since we did not specify the precedence of @samp{/}, the
8208sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
8209NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
8210NUM}, which corresponds to reducing rule 1.
8211
eb45ef3b 8212Because in deterministic parsing a single decision can be made, Bison
ec3bc396
AD
8213arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
8214Shift/Reduce Conflicts}. Discarded actions are reported in between
8215square brackets.
8216
8217Note that all the previous states had a single possible action: either
8218shifting the next token and going to the corresponding state, or
8219reducing a single rule. In the other cases, i.e., when shifting
8220@emph{and} reducing is possible or when @emph{several} reductions are
742e4900
JD
8221possible, the lookahead is required to select the action. State 8 is
8222one such state: if the lookahead is @samp{*} or @samp{/} then the action
ec3bc396
AD
8223is shifting, otherwise the action is reducing rule 1. In other words,
8224the first two items, corresponding to rule 1, are not eligible when the
742e4900 8225lookahead token is @samp{*}, since we specified that @samp{*} has higher
8dd162d3 8226precedence than @samp{+}. More generally, some items are eligible only
742e4900
JD
8227with some set of possible lookahead tokens. When run with
8228@option{--report=lookahead}, Bison specifies these lookahead tokens:
ec3bc396
AD
8229
8230@example
8231state 8
8232
88c78747 8233 exp -> exp . '+' exp (rule 1)
ec3bc396
AD
8234 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
8235 exp -> exp . '-' exp (rule 2)
8236 exp -> exp . '*' exp (rule 3)
8237 exp -> exp . '/' exp (rule 4)
8238
8239 '*' shift, and go to state 6
8240 '/' shift, and go to state 7
8241
8242 '/' [reduce using rule 1 (exp)]
8243 $default reduce using rule 1 (exp)
8244@end example
8245
8246The remaining states are similar:
8247
8248@example
8249state 9
8250
8251 exp -> exp . '+' exp (rule 1)
8252 exp -> exp . '-' exp (rule 2)
8253 exp -> exp '-' exp . (rule 2)
8254 exp -> exp . '*' exp (rule 3)
8255 exp -> exp . '/' exp (rule 4)
8256
2a8d363a
AD
8257 '*' shift, and go to state 6
8258 '/' shift, and go to state 7
ec3bc396 8259
2a8d363a
AD
8260 '/' [reduce using rule 2 (exp)]
8261 $default reduce using rule 2 (exp)
ec3bc396
AD
8262
8263state 10
8264
8265 exp -> exp . '+' exp (rule 1)
8266 exp -> exp . '-' exp (rule 2)
8267 exp -> exp . '*' exp (rule 3)
8268 exp -> exp '*' exp . (rule 3)
8269 exp -> exp . '/' exp (rule 4)
8270
2a8d363a 8271 '/' shift, and go to state 7
ec3bc396 8272
2a8d363a
AD
8273 '/' [reduce using rule 3 (exp)]
8274 $default reduce using rule 3 (exp)
ec3bc396
AD
8275
8276state 11
8277
8278 exp -> exp . '+' exp (rule 1)
8279 exp -> exp . '-' exp (rule 2)
8280 exp -> exp . '*' exp (rule 3)
8281 exp -> exp . '/' exp (rule 4)
8282 exp -> exp '/' exp . (rule 4)
8283
2a8d363a
AD
8284 '+' shift, and go to state 4
8285 '-' shift, and go to state 5
8286 '*' shift, and go to state 6
8287 '/' shift, and go to state 7
ec3bc396 8288
2a8d363a
AD
8289 '+' [reduce using rule 4 (exp)]
8290 '-' [reduce using rule 4 (exp)]
8291 '*' [reduce using rule 4 (exp)]
8292 '/' [reduce using rule 4 (exp)]
8293 $default reduce using rule 4 (exp)
ec3bc396
AD
8294@end example
8295
8296@noindent
fa7e68c3
PE
8297Observe that state 11 contains conflicts not only due to the lack of
8298precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
8299@samp{*}, but also because the
ec3bc396
AD
8300associativity of @samp{/} is not specified.
8301
8302
8303@node Tracing
8304@section Tracing Your Parser
bfa74976
RS
8305@findex yydebug
8306@cindex debugging
8307@cindex tracing the parser
8308
8309If a Bison grammar compiles properly but doesn't do what you want when it
8310runs, the @code{yydebug} parser-trace feature can help you figure out why.
8311
3ded9a63
AD
8312There are several means to enable compilation of trace facilities:
8313
8314@table @asis
8315@item the macro @code{YYDEBUG}
8316@findex YYDEBUG
8317Define the macro @code{YYDEBUG} to a nonzero value when you compile the
8a4281b9 8318parser. This is compliant with POSIX Yacc. You could use
3ded9a63
AD
8319@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
8320YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
8321Prologue}).
8322
8323@item the option @option{-t}, @option{--debug}
8324Use the @samp{-t} option when you run Bison (@pxref{Invocation,
8a4281b9 8325,Invoking Bison}). This is POSIX compliant too.
3ded9a63
AD
8326
8327@item the directive @samp{%debug}
8328@findex %debug
fa819509
AD
8329Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison Declaration
8330Summary}). This Bison extension is maintained for backward
8331compatibility with previous versions of Bison.
8332
8333@item the variable @samp{parse.trace}
8334@findex %define parse.trace
35c1e5f0
JD
8335Add the @samp{%define parse.trace} directive (@pxref{%define
8336Summary,,parse.trace}), or pass the @option{-Dparse.trace} option
fa819509 8337(@pxref{Bison Options}). This is a Bison extension, which is especially
35c1e5f0
JD
8338useful for languages that don't use a preprocessor. Unless POSIX and Yacc
8339portability matter to you, this is the preferred solution.
3ded9a63
AD
8340@end table
8341
fa819509 8342We suggest that you always enable the trace option so that debugging is
3ded9a63 8343always possible.
bfa74976 8344
02a81e05 8345The trace facility outputs messages with macro calls of the form
e2742e46 8346@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
f57a7536 8347@var{format} and @var{args} are the usual @code{printf} format and variadic
4947ebdb
PE
8348arguments. If you define @code{YYDEBUG} to a nonzero value but do not
8349define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
9c437126 8350and @code{YYFPRINTF} is defined to @code{fprintf}.
bfa74976
RS
8351
8352Once you have compiled the program with trace facilities, the way to
8353request a trace is to store a nonzero value in the variable @code{yydebug}.
8354You can do this by making the C code do it (in @code{main}, perhaps), or
8355you can alter the value with a C debugger.
8356
8357Each step taken by the parser when @code{yydebug} is nonzero produces a
8358line or two of trace information, written on @code{stderr}. The trace
8359messages tell you these things:
8360
8361@itemize @bullet
8362@item
8363Each time the parser calls @code{yylex}, what kind of token was read.
8364
8365@item
8366Each time a token is shifted, the depth and complete contents of the
8367state stack (@pxref{Parser States}).
8368
8369@item
8370Each time a rule is reduced, which rule it is, and the complete contents
8371of the state stack afterward.
8372@end itemize
8373
8374To make sense of this information, it helps to refer to the listing file
704a47c4
AD
8375produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
8376Bison}). This file shows the meaning of each state in terms of
8377positions in various rules, and also what each state will do with each
8378possible input token. As you read the successive trace messages, you
8379can see that the parser is functioning according to its specification in
8380the listing file. Eventually you will arrive at the place where
8381something undesirable happens, and you will see which parts of the
8382grammar are to blame.
bfa74976 8383
ff7571c0
JD
8384The parser implementation file is a C program and you can use C
8385debuggers on it, but it's not easy to interpret what it is doing. The
8386parser function is a finite-state machine interpreter, and aside from
8387the actions it executes the same code over and over. Only the values
8388of variables show where in the grammar it is working.
bfa74976
RS
8389
8390@findex YYPRINT
8391The debugging information normally gives the token type of each token
8392read, but not its semantic value. You can optionally define a macro
8393named @code{YYPRINT} to provide a way to print the value. If you define
8394@code{YYPRINT}, it should take three arguments. The parser will pass a
8395standard I/O stream, the numeric code for the token type, and the token
8396value (from @code{yylval}).
8397
8398Here is an example of @code{YYPRINT} suitable for the multi-function
f5f419de 8399calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}):
bfa74976
RS
8400
8401@smallexample
38a92d50
PE
8402%@{
8403 static void print_token_value (FILE *, int, YYSTYPE);
8404 #define YYPRINT(file, type, value) print_token_value (file, type, value)
8405%@}
8406
8407@dots{} %% @dots{} %% @dots{}
bfa74976
RS
8408
8409static void
831d3c99 8410print_token_value (FILE *file, int type, YYSTYPE value)
bfa74976
RS
8411@{
8412 if (type == VAR)
d3c4e709 8413 fprintf (file, "%s", value.tptr->name);
bfa74976 8414 else if (type == NUM)
d3c4e709 8415 fprintf (file, "%d", value.val);
bfa74976
RS
8416@}
8417@end smallexample
8418
ec3bc396
AD
8419@c ================================================= Invoking Bison
8420
342b8b6e 8421@node Invocation
bfa74976
RS
8422@chapter Invoking Bison
8423@cindex invoking Bison
8424@cindex Bison invocation
8425@cindex options for invoking Bison
8426
8427The usual way to invoke Bison is as follows:
8428
8429@example
8430bison @var{infile}
8431@end example
8432
8433Here @var{infile} is the grammar file name, which usually ends in
ff7571c0
JD
8434@samp{.y}. The parser implementation file's name is made by replacing
8435the @samp{.y} with @samp{.tab.c} and removing any leading directory.
8436Thus, the @samp{bison foo.y} file name yields @file{foo.tab.c}, and
8437the @samp{bison hack/foo.y} file name yields @file{foo.tab.c}. It's
8438also possible, in case you are writing C++ code instead of C in your
8439grammar file, to name it @file{foo.ypp} or @file{foo.y++}. Then, the
8440output files will take an extension like the given one as input
8441(respectively @file{foo.tab.cpp} and @file{foo.tab.c++}). This
8442feature takes effect with all options that manipulate file names like
234a3be3
AD
8443@samp{-o} or @samp{-d}.
8444
8445For example :
8446
8447@example
8448bison -d @var{infile.yxx}
8449@end example
84163231 8450@noindent
72d2299c 8451will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
234a3be3
AD
8452
8453@example
b56471a6 8454bison -d -o @var{output.c++} @var{infile.y}
234a3be3 8455@end example
84163231 8456@noindent
234a3be3
AD
8457will produce @file{output.c++} and @file{outfile.h++}.
8458
8a4281b9 8459For compatibility with POSIX, the standard Bison
397ec073
PE
8460distribution also contains a shell script called @command{yacc} that
8461invokes Bison with the @option{-y} option.
8462
bfa74976 8463@menu
13863333 8464* Bison Options:: All the options described in detail,
c827f760 8465 in alphabetical order by short options.
bfa74976 8466* Option Cross Key:: Alphabetical list of long options.
93dd49ab 8467* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
bfa74976
RS
8468@end menu
8469
342b8b6e 8470@node Bison Options
bfa74976
RS
8471@section Bison Options
8472
8473Bison supports both traditional single-letter options and mnemonic long
8474option names. Long option names are indicated with @samp{--} instead of
8475@samp{-}. Abbreviations for option names are allowed as long as they
8476are unique. When a long option takes an argument, like
8477@samp{--file-prefix}, connect the option name and the argument with
8478@samp{=}.
8479
8480Here is a list of options that can be used with Bison, alphabetized by
8481short option. It is followed by a cross key alphabetized by long
8482option.
8483
89cab50d
AD
8484@c Please, keep this ordered as in `bison --help'.
8485@noindent
8486Operations modes:
8487@table @option
8488@item -h
8489@itemx --help
8490Print a summary of the command-line options to Bison and exit.
bfa74976 8491
89cab50d
AD
8492@item -V
8493@itemx --version
8494Print the version number of Bison and exit.
bfa74976 8495
f7ab6a50
PE
8496@item --print-localedir
8497Print the name of the directory containing locale-dependent data.
8498
a0de5091
JD
8499@item --print-datadir
8500Print the name of the directory containing skeletons and XSLT.
8501
89cab50d
AD
8502@item -y
8503@itemx --yacc
ff7571c0
JD
8504Act more like the traditional Yacc command. This can cause different
8505diagnostics to be generated, and may change behavior in other minor
8506ways. Most importantly, imitate Yacc's output file name conventions,
8507so that the parser implementation file is called @file{y.tab.c}, and
8508the other outputs are called @file{y.output} and @file{y.tab.h}.
8509Also, if generating a deterministic parser in C, generate
8510@code{#define} statements in addition to an @code{enum} to associate
8511token numbers with token names. Thus, the following shell script can
8512substitute for Yacc, and the Bison distribution contains such a script
8513for compatibility with POSIX:
bfa74976 8514
89cab50d 8515@example
397ec073 8516#! /bin/sh
26e06a21 8517bison -y "$@@"
89cab50d 8518@end example
54662697
PE
8519
8520The @option{-y}/@option{--yacc} option is intended for use with
8521traditional Yacc grammars. If your grammar uses a Bison extension
8522like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
8523this option is specified.
8524
1d5b3c08
JD
8525@item -W [@var{category}]
8526@itemx --warnings[=@var{category}]
118d4978
AD
8527Output warnings falling in @var{category}. @var{category} can be one
8528of:
8529@table @code
8530@item midrule-values
8e55b3aa
JD
8531Warn about mid-rule values that are set but not used within any of the actions
8532of the parent rule.
8533For example, warn about unused @code{$2} in:
118d4978
AD
8534
8535@example
8536exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
8537@end example
8538
8e55b3aa
JD
8539Also warn about mid-rule values that are used but not set.
8540For example, warn about unset @code{$$} in the mid-rule action in:
118d4978
AD
8541
8542@example
8543 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
8544@end example
8545
8546These warnings are not enabled by default since they sometimes prove to
8547be false alarms in existing grammars employing the Yacc constructs
8e55b3aa 8548@code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
118d4978
AD
8549
8550
8551@item yacc
8a4281b9 8552Incompatibilities with POSIX Yacc.
118d4978
AD
8553
8554@item all
8e55b3aa 8555All the warnings.
118d4978 8556@item none
8e55b3aa 8557Turn off all the warnings.
118d4978 8558@item error
8e55b3aa 8559Treat warnings as errors.
118d4978
AD
8560@end table
8561
8562A category can be turned off by prefixing its name with @samp{no-}. For
93d7dde9 8563instance, @option{-Wno-yacc} will hide the warnings about
8a4281b9 8564POSIX Yacc incompatibilities.
89cab50d
AD
8565@end table
8566
8567@noindent
8568Tuning the parser:
8569
8570@table @option
8571@item -t
8572@itemx --debug
ff7571c0
JD
8573In the parser implementation file, define the macro @code{YYDEBUG} to
85741 if it is not already defined, so that the debugging facilities are
8575compiled. @xref{Tracing, ,Tracing Your Parser}.
89cab50d 8576
58697c6d
AD
8577@item -D @var{name}[=@var{value}]
8578@itemx --define=@var{name}[=@var{value}]
17aed602 8579@itemx -F @var{name}[=@var{value}]
de5ab940
JD
8580@itemx --force-define=@var{name}[=@var{value}]
8581Each of these is equivalent to @samp{%define @var{name} "@var{value}"}
35c1e5f0 8582(@pxref{%define Summary}) except that Bison processes multiple
de5ab940
JD
8583definitions for the same @var{name} as follows:
8584
8585@itemize
8586@item
0b6d43c5
JD
8587Bison quietly ignores all command-line definitions for @var{name} except
8588the last.
de5ab940 8589@item
0b6d43c5
JD
8590If that command-line definition is specified by a @code{-D} or
8591@code{--define}, Bison reports an error for any @code{%define}
8592definition for @var{name}.
de5ab940 8593@item
0b6d43c5
JD
8594If that command-line definition is specified by a @code{-F} or
8595@code{--force-define} instead, Bison quietly ignores all @code{%define}
8596definitions for @var{name}.
8597@item
8598Otherwise, Bison reports an error if there are multiple @code{%define}
8599definitions for @var{name}.
de5ab940
JD
8600@end itemize
8601
8602You should avoid using @code{-F} and @code{--force-define} in your
ff7571c0
JD
8603make files unless you are confident that it is safe to quietly ignore
8604any conflicting @code{%define} that may be added to the grammar file.
58697c6d 8605
0e021770
PE
8606@item -L @var{language}
8607@itemx --language=@var{language}
8608Specify the programming language for the generated parser, as if
8609@code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
59da312b 8610Summary}). Currently supported languages include C, C++, and Java.
e6e704dc 8611@var{language} is case-insensitive.
0e021770 8612
ed4d67dc
JD
8613This option is experimental and its effect may be modified in future
8614releases.
8615
89cab50d 8616@item --locations
d8988b2f 8617Pretend that @code{%locations} was specified. @xref{Decl Summary}.
89cab50d
AD
8618
8619@item -p @var{prefix}
8620@itemx --name-prefix=@var{prefix}
02975b9a 8621Pretend that @code{%name-prefix "@var{prefix}"} was specified.
d8988b2f 8622@xref{Decl Summary}.
bfa74976
RS
8623
8624@item -l
8625@itemx --no-lines
ff7571c0
JD
8626Don't put any @code{#line} preprocessor commands in the parser
8627implementation file. Ordinarily Bison puts them in the parser
8628implementation file so that the C compiler and debuggers will
8629associate errors with your source file, the grammar file. This option
8630causes them to associate errors with the parser implementation file,
8631treating it as an independent source file in its own right.
bfa74976 8632
e6e704dc
JD
8633@item -S @var{file}
8634@itemx --skeleton=@var{file}
a7867f53 8635Specify the skeleton to use, similar to @code{%skeleton}
e6e704dc
JD
8636(@pxref{Decl Summary, , Bison Declaration Summary}).
8637
ed4d67dc
JD
8638@c You probably don't need this option unless you are developing Bison.
8639@c You should use @option{--language} if you want to specify the skeleton for a
8640@c different language, because it is clearer and because it will always
8641@c choose the correct skeleton for non-deterministic or push parsers.
e6e704dc 8642
a7867f53
JD
8643If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
8644file in the Bison installation directory.
8645If it does, @var{file} is an absolute file name or a file name relative to the
8646current working directory.
8647This is similar to how most shells resolve commands.
8648
89cab50d
AD
8649@item -k
8650@itemx --token-table
d8988b2f 8651Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
89cab50d 8652@end table
bfa74976 8653
89cab50d
AD
8654@noindent
8655Adjust the output:
bfa74976 8656
89cab50d 8657@table @option
8e55b3aa 8658@item --defines[=@var{file}]
d8988b2f 8659Pretend that @code{%defines} was specified, i.e., write an extra output
6deb4447 8660file containing macro definitions for the token type names defined in
4bfd5e4e 8661the grammar, as well as a few other declarations. @xref{Decl Summary}.
931c7513 8662
8e55b3aa
JD
8663@item -d
8664This is the same as @code{--defines} except @code{-d} does not accept a
8665@var{file} argument since POSIX Yacc requires that @code{-d} can be bundled
8666with other short options.
342b8b6e 8667
89cab50d
AD
8668@item -b @var{file-prefix}
8669@itemx --file-prefix=@var{prefix}
9c437126 8670Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
72d2299c 8671for all Bison output file names. @xref{Decl Summary}.
bfa74976 8672
ec3bc396
AD
8673@item -r @var{things}
8674@itemx --report=@var{things}
8675Write an extra output file containing verbose description of the comma
8676separated list of @var{things} among:
8677
8678@table @code
8679@item state
8680Description of the grammar, conflicts (resolved and unresolved), and
eb45ef3b 8681parser's automaton.
ec3bc396 8682
742e4900 8683@item lookahead
ec3bc396 8684Implies @code{state} and augments the description of the automaton with
742e4900 8685each rule's lookahead set.
ec3bc396
AD
8686
8687@item itemset
8688Implies @code{state} and augments the description of the automaton with
8689the full set of items for each state, instead of its core only.
8690@end table
8691
1bb2bd75
JD
8692@item --report-file=@var{file}
8693Specify the @var{file} for the verbose description.
8694
bfa74976
RS
8695@item -v
8696@itemx --verbose
9c437126 8697Pretend that @code{%verbose} was specified, i.e., write an extra output
6deb4447 8698file containing verbose descriptions of the grammar and
72d2299c 8699parser. @xref{Decl Summary}.
bfa74976 8700
fa4d969f
PE
8701@item -o @var{file}
8702@itemx --output=@var{file}
ff7571c0 8703Specify the @var{file} for the parser implementation file.
bfa74976 8704
fa4d969f 8705The other output files' names are constructed from @var{file} as
d8988b2f 8706described under the @samp{-v} and @samp{-d} options.
342b8b6e 8707
a7c09cba 8708@item -g [@var{file}]
8e55b3aa 8709@itemx --graph[=@var{file}]
eb45ef3b 8710Output a graphical representation of the parser's
35fe0834 8711automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
8a4281b9 8712@uref{http://www.graphviz.org/doc/info/lang.html, DOT} format.
8e55b3aa
JD
8713@code{@var{file}} is optional.
8714If omitted and the grammar file is @file{foo.y}, the output file will be
8715@file{foo.dot}.
59da312b 8716
a7c09cba 8717@item -x [@var{file}]
8e55b3aa 8718@itemx --xml[=@var{file}]
eb45ef3b 8719Output an XML report of the parser's automaton computed by Bison.
8e55b3aa 8720@code{@var{file}} is optional.
59da312b
JD
8721If omitted and the grammar file is @file{foo.y}, the output file will be
8722@file{foo.xml}.
8723(The current XML schema is experimental and may evolve.
8724More user feedback will help to stabilize it.)
bfa74976
RS
8725@end table
8726
342b8b6e 8727@node Option Cross Key
bfa74976
RS
8728@section Option Cross Key
8729
8730Here is a list of options, alphabetized by long option, to help you find
de5ab940 8731the corresponding short option and directive.
bfa74976 8732
de5ab940 8733@multitable {@option{--force-define=@var{name}[=@var{value}]}} {@option{-F @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}}
a7c09cba 8734@headitem Long Option @tab Short Option @tab Bison Directive
f4101aa6 8735@include cross-options.texi
aa08666d 8736@end multitable
bfa74976 8737
93dd49ab
PE
8738@node Yacc Library
8739@section Yacc Library
8740
8741The Yacc library contains default implementations of the
8742@code{yyerror} and @code{main} functions. These default
8a4281b9 8743implementations are normally not useful, but POSIX requires
93dd49ab
PE
8744them. To use the Yacc library, link your program with the
8745@option{-ly} option. Note that Bison's implementation of the Yacc
8a4281b9 8746library is distributed under the terms of the GNU General
93dd49ab
PE
8747Public License (@pxref{Copying}).
8748
8749If you use the Yacc library's @code{yyerror} function, you should
8750declare @code{yyerror} as follows:
8751
8752@example
8753int yyerror (char const *);
8754@end example
8755
8756Bison ignores the @code{int} value returned by this @code{yyerror}.
8757If you use the Yacc library's @code{main} function, your
8758@code{yyparse} function should have the following type signature:
8759
8760@example
8761int yyparse (void);
8762@end example
8763
12545799
AD
8764@c ================================================= C++ Bison
8765
8405b70c
PB
8766@node Other Languages
8767@chapter Parsers Written In Other Languages
12545799
AD
8768
8769@menu
8770* C++ Parsers:: The interface to generate C++ parser classes
8405b70c 8771* Java Parsers:: The interface to generate Java parser classes
12545799
AD
8772@end menu
8773
8774@node C++ Parsers
8775@section C++ Parsers
8776
8777@menu
8778* C++ Bison Interface:: Asking for C++ parser generation
8779* C++ Semantic Values:: %union vs. C++
8780* C++ Location Values:: The position and location classes
8781* C++ Parser Interface:: Instantiating and running the parser
8782* C++ Scanner Interface:: Exchanges between yylex and parse
8405b70c 8783* A Complete C++ Example:: Demonstrating their use
12545799
AD
8784@end menu
8785
8786@node C++ Bison Interface
8787@subsection C++ Bison Interface
ed4d67dc 8788@c - %skeleton "lalr1.cc"
12545799
AD
8789@c - Always pure
8790@c - initial action
8791
eb45ef3b 8792The C++ deterministic parser is selected using the skeleton directive,
86e5b440
AD
8793@samp{%skeleton "lalr1.cc"}, or the synonymous command-line option
8794@option{--skeleton=lalr1.cc}.
e6e704dc 8795@xref{Decl Summary}.
0e021770 8796
793fbca5
JD
8797When run, @command{bison} will create several entities in the @samp{yy}
8798namespace.
67501061 8799@findex %define api.namespace
35c1e5f0
JD
8800Use the @samp{%define api.namespace} directive to change the namespace name,
8801see @ref{%define Summary,,api.namespace}. The various classes are generated
8802in the following files:
aa08666d 8803
12545799
AD
8804@table @file
8805@item position.hh
8806@itemx location.hh
8807The definition of the classes @code{position} and @code{location},
3cdc21cf 8808used for location tracking when enabled. @xref{C++ Location Values}.
12545799
AD
8809
8810@item stack.hh
8811An auxiliary class @code{stack} used by the parser.
8812
fa4d969f
PE
8813@item @var{file}.hh
8814@itemx @var{file}.cc
ff7571c0 8815(Assuming the extension of the grammar file was @samp{.yy}.) The
cd8b5791
AD
8816declaration and implementation of the C++ parser class. The basename
8817and extension of these two files follow the same rules as with regular C
8818parsers (@pxref{Invocation}).
12545799 8819
cd8b5791
AD
8820The header is @emph{mandatory}; you must either pass
8821@option{-d}/@option{--defines} to @command{bison}, or use the
12545799
AD
8822@samp{%defines} directive.
8823@end table
8824
8825All these files are documented using Doxygen; run @command{doxygen}
8826for a complete and accurate documentation.
8827
8828@node C++ Semantic Values
8829@subsection C++ Semantic Values
8830@c - No objects in unions
178e123e 8831@c - YYSTYPE
12545799
AD
8832@c - Printer and destructor
8833
3cdc21cf
AD
8834Bison supports two different means to handle semantic values in C++. One is
8835alike the C interface, and relies on unions (@pxref{C++ Unions}). As C++
8836practitioners know, unions are inconvenient in C++, therefore another
8837approach is provided, based on variants (@pxref{C++ Variants}).
8838
8839@menu
8840* C++ Unions:: Semantic values cannot be objects
8841* C++ Variants:: Using objects as semantic values
8842@end menu
8843
8844@node C++ Unions
8845@subsubsection C++ Unions
8846
12545799
AD
8847The @code{%union} directive works as for C, see @ref{Union Decl, ,The
8848Collection of Value Types}. In particular it produces a genuine
3cdc21cf 8849@code{union}, which have a few specific features in C++.
12545799
AD
8850@itemize @minus
8851@item
fb9712a9
AD
8852The type @code{YYSTYPE} is defined but its use is discouraged: rather
8853you should refer to the parser's encapsulated type
8854@code{yy::parser::semantic_type}.
12545799
AD
8855@item
8856Non POD (Plain Old Data) types cannot be used. C++ forbids any
8857instance of classes with constructors in unions: only @emph{pointers}
8858to such objects are allowed.
8859@end itemize
8860
8861Because objects have to be stored via pointers, memory is not
8862reclaimed automatically: using the @code{%destructor} directive is the
8863only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
8864Symbols}.
8865
3cdc21cf
AD
8866@node C++ Variants
8867@subsubsection C++ Variants
8868
8869Starting with version 2.6, Bison provides a @emph{variant} based
8870implementation of semantic values for C++. This alleviates all the
8871limitations reported in the previous section, and in particular, object
8872types can be used without pointers.
8873
8874To enable variant-based semantic values, set @code{%define} variable
35c1e5f0 8875@code{variant} (@pxref{%define Summary,, variant}). Once this defined,
3cdc21cf
AD
8876@code{%union} is ignored, and instead of using the name of the fields of the
8877@code{%union} to ``type'' the symbols, use genuine types.
8878
8879For instance, instead of
8880
8881@example
8882%union
8883@{
8884 int ival;
8885 std::string* sval;
8886@}
8887%token <ival> NUMBER;
8888%token <sval> STRING;
8889@end example
8890
8891@noindent
8892write
8893
8894@example
8895%token <int> NUMBER;
8896%token <std::string> STRING;
8897@end example
8898
8899@code{STRING} is no longer a pointer, which should fairly simplify the user
8900actions in the grammar and in the scanner (in particular the memory
8901management).
8902
8903Since C++ features destructors, and since it is customary to specialize
8904@code{operator<<} to support uniform printing of values, variants also
8905typically simplify Bison printers and destructors.
8906
8907Variants are stricter than unions. When based on unions, you may play any
8908dirty game with @code{yylval}, say storing an @code{int}, reading a
8909@code{char*}, and then storing a @code{double} in it. This is no longer
8910possible with variants: they must be initialized, then assigned to, and
8911eventually, destroyed.
8912
8913@deftypemethod {semantic_type} {T&} build<T> ()
8914Initialize, but leave empty. Returns the address where the actual value may
8915be stored. Requires that the variant was not initialized yet.
8916@end deftypemethod
8917
8918@deftypemethod {semantic_type} {T&} build<T> (const T& @var{t})
8919Initialize, and copy-construct from @var{t}.
8920@end deftypemethod
8921
8922
8923@strong{Warning}: We do not use Boost.Variant, for two reasons. First, it
8924appeared unacceptable to require Boost on the user's machine (i.e., the
8925machine on which the generated parser will be compiled, not the machine on
8926which @command{bison} was run). Second, for each possible semantic value,
8927Boost.Variant not only stores the value, but also a tag specifying its
8928type. But the parser already ``knows'' the type of the semantic value, so
8929that would be duplicating the information.
8930
8931Therefore we developed light-weight variants whose type tag is external (so
8932they are really like @code{unions} for C++ actually). But our code is much
8933less mature that Boost.Variant. So there is a number of limitations in
8934(the current implementation of) variants:
8935@itemize
8936@item
8937Alignment must be enforced: values should be aligned in memory according to
8938the most demanding type. Computing the smallest alignment possible requires
8939meta-programming techniques that are not currently implemented in Bison, and
8940therefore, since, as far as we know, @code{double} is the most demanding
8941type on all platforms, alignments are enforced for @code{double} whatever
8942types are actually used. This may waste space in some cases.
8943
8944@item
8945Our implementation is not conforming with strict aliasing rules. Alias
8946analysis is a technique used in optimizing compilers to detect when two
8947pointers are disjoint (they cannot ``meet''). Our implementation breaks
8948some of the rules that G++ 4.4 uses in its alias analysis, so @emph{strict
8949alias analysis must be disabled}. Use the option
8950@option{-fno-strict-aliasing} to compile the generated parser.
8951
8952@item
8953There might be portability issues we are not aware of.
8954@end itemize
8955
a6ca4ce2 8956As far as we know, these limitations @emph{can} be alleviated. All it takes
3cdc21cf 8957is some time and/or some talented C++ hacker willing to contribute to Bison.
12545799
AD
8958
8959@node C++ Location Values
8960@subsection C++ Location Values
8961@c - %locations
8962@c - class Position
8963@c - class Location
16dc6a9e 8964@c - %define filename_type "const symbol::Symbol"
12545799
AD
8965
8966When the directive @code{%locations} is used, the C++ parser supports
8967location tracking, see @ref{Locations, , Locations Overview}. Two
8968auxiliary classes define a @code{position}, a single point in a file,
8969and a @code{location}, a range composed of a pair of
8970@code{position}s (possibly spanning several files).
8971
fa4d969f 8972@deftypemethod {position} {std::string*} file
12545799
AD
8973The name of the file. It will always be handled as a pointer, the
8974parser will never duplicate nor deallocate it. As an experimental
8975feature you may change it to @samp{@var{type}*} using @samp{%define
16dc6a9e 8976filename_type "@var{type}"}.
12545799
AD
8977@end deftypemethod
8978
8979@deftypemethod {position} {unsigned int} line
8980The line, starting at 1.
8981@end deftypemethod
8982
8983@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
8984Advance by @var{height} lines, resetting the column number.
8985@end deftypemethod
8986
8987@deftypemethod {position} {unsigned int} column
8988The column, starting at 0.
8989@end deftypemethod
8990
8991@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
8992Advance by @var{width} columns, without changing the line number.
8993@end deftypemethod
8994
8995@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
8996@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
8997@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
8998@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
8999Various forms of syntactic sugar for @code{columns}.
9000@end deftypemethod
9001
9002@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
9003Report @var{p} on @var{o} like this:
fa4d969f
PE
9004@samp{@var{file}:@var{line}.@var{column}}, or
9005@samp{@var{line}.@var{column}} if @var{file} is null.
12545799
AD
9006@end deftypemethod
9007
9008@deftypemethod {location} {position} begin
9009@deftypemethodx {location} {position} end
9010The first, inclusive, position of the range, and the first beyond.
9011@end deftypemethod
9012
9013@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
9014@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
9015Advance the @code{end} position.
9016@end deftypemethod
9017
9018@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
9019@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
9020@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
9021Various forms of syntactic sugar.
9022@end deftypemethod
9023
9024@deftypemethod {location} {void} step ()
9025Move @code{begin} onto @code{end}.
9026@end deftypemethod
9027
9028
9029@node C++ Parser Interface
9030@subsection C++ Parser Interface
9031@c - define parser_class_name
9032@c - Ctor
9033@c - parse, error, set_debug_level, debug_level, set_debug_stream,
9034@c debug_stream.
9035@c - Reporting errors
9036
9037The output files @file{@var{output}.hh} and @file{@var{output}.cc}
9038declare and define the parser class in the namespace @code{yy}. The
9039class name defaults to @code{parser}, but may be changed using
16dc6a9e 9040@samp{%define parser_class_name "@var{name}"}. The interface of
9d9b8b70 9041this class is detailed below. It can be extended using the
12545799
AD
9042@code{%parse-param} feature: its semantics is slightly changed since
9043it describes an additional member of the parser class, and an
9044additional argument for its constructor.
9045
3cdc21cf
AD
9046@defcv {Type} {parser} {semantic_type}
9047@defcvx {Type} {parser} {location_type}
9048The types for semantic values and locations (if enabled).
9049@end defcv
9050
86e5b440
AD
9051@defcv {Type} {parser} {token}
9052A structure that contains (only) the definition of the tokens as the
9053@code{yytokentype} enumeration. To refer to the token @code{FOO}, the
9054scanner should use @code{yy::parser::token::FOO}. The scanner can use
9055@samp{typedef yy::parser::token token;} to ``import'' the token enumeration
9056(@pxref{Calc++ Scanner}).
9057@end defcv
9058
3cdc21cf
AD
9059@defcv {Type} {parser} {syntax_error}
9060This class derives from @code{std::runtime_error}. Throw instances of it
9061from user actions to raise parse errors. This is equivalent with first
9062invoking @code{error} to report the location and message of the syntax
9063error, and then to invoke @code{YYERROR} to enter the error-recovery mode.
9064But contrary to @code{YYERROR} which can only be invoked from user actions
9065(i.e., written in the action itself), the exception can be thrown from
9066function invoked from the user action.
8a0adb01 9067@end defcv
12545799
AD
9068
9069@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
9070Build a new parser object. There are no arguments by default, unless
9071@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
9072@end deftypemethod
9073
3cdc21cf
AD
9074@deftypemethod {syntax_error} {} syntax_error (const location_type& @var{l}, const std::string& @var{m})
9075@deftypemethodx {syntax_error} {} syntax_error (const std::string& @var{m})
9076Instantiate a syntax-error exception.
9077@end deftypemethod
9078
12545799
AD
9079@deftypemethod {parser} {int} parse ()
9080Run the syntactic analysis, and return 0 on success, 1 otherwise.
9081@end deftypemethod
9082
9083@deftypemethod {parser} {std::ostream&} debug_stream ()
9084@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
9085Get or set the stream used for tracing the parsing. It defaults to
9086@code{std::cerr}.
9087@end deftypemethod
9088
9089@deftypemethod {parser} {debug_level_type} debug_level ()
9090@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
9091Get or set the tracing level. Currently its value is either 0, no trace,
9d9b8b70 9092or nonzero, full tracing.
12545799
AD
9093@end deftypemethod
9094
9095@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
3cdc21cf 9096@deftypemethodx {parser} {void} error (const std::string& @var{m})
12545799
AD
9097The definition for this member function must be supplied by the user:
9098the parser uses it to report a parser error occurring at @var{l},
3cdc21cf
AD
9099described by @var{m}. If location tracking is not enabled, the second
9100signature is used.
12545799
AD
9101@end deftypemethod
9102
9103
9104@node C++ Scanner Interface
9105@subsection C++ Scanner Interface
9106@c - prefix for yylex.
9107@c - Pure interface to yylex
9108@c - %lex-param
9109
9110The parser invokes the scanner by calling @code{yylex}. Contrary to C
9111parsers, C++ parsers are always pure: there is no point in using the
3cdc21cf
AD
9112@samp{%define api.pure} directive. The actual interface with @code{yylex}
9113depends whether you use unions, or variants.
12545799 9114
3cdc21cf
AD
9115@menu
9116* Split Symbols:: Passing symbols as two/three components
9117* Complete Symbols:: Making symbols a whole
9118@end menu
9119
9120@node Split Symbols
9121@subsubsection Split Symbols
9122
9123Therefore the interface is as follows.
9124
86e5b440
AD
9125@deftypemethod {parser} {int} yylex (semantic_type* @var{yylval}, location_type* @var{yylloc}, @var{type1} @var{arg1}, ...)
9126@deftypemethodx {parser} {int} yylex (semantic_type* @var{yylval}, @var{type1} @var{arg1}, ...)
3cdc21cf
AD
9127Return the next token. Its type is the return value, its semantic value and
9128location (if enabled) being @var{yylval} and @var{yylloc}. Invocations of
12545799
AD
9129@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
9130@end deftypemethod
9131
3cdc21cf
AD
9132Note that when using variants, the interface for @code{yylex} is the same,
9133but @code{yylval} is handled differently.
9134
9135Regular union-based code in Lex scanner typically look like:
9136
9137@example
9138[0-9]+ @{
9139 yylval.ival = text_to_int (yytext);
9140 return yy::parser::INTEGER;
9141 @}
9142[a-z]+ @{
9143 yylval.sval = new std::string (yytext);
9144 return yy::parser::IDENTIFIER;
9145 @}
9146@end example
9147
9148Using variants, @code{yylval} is already constructed, but it is not
9149initialized. So the code would look like:
9150
9151@example
9152[0-9]+ @{
9153 yylval.build<int>() = text_to_int (yytext);
9154 return yy::parser::INTEGER;
9155 @}
9156[a-z]+ @{
9157 yylval.build<std::string> = yytext;
9158 return yy::parser::IDENTIFIER;
9159 @}
9160@end example
9161
9162@noindent
9163or
9164
9165@example
9166[0-9]+ @{
9167 yylval.build(text_to_int (yytext));
9168 return yy::parser::INTEGER;
9169 @}
9170[a-z]+ @{
9171 yylval.build(yytext);
9172 return yy::parser::IDENTIFIER;
9173 @}
9174@end example
9175
9176
9177@node Complete Symbols
9178@subsubsection Complete Symbols
9179
9180If you specified both @code{%define variant} and @code{%define lex_symbol},
9181the @code{parser} class also defines the class @code{parser::symbol_type}
9182which defines a @emph{complete} symbol, aggregating its type (i.e., the
9183traditional value returned by @code{yylex}), its semantic value (i.e., the
9184value passed in @code{yylval}, and possibly its location (@code{yylloc}).
9185
9186@deftypemethod {symbol_type} {} symbol_type (token_type @var{type}, const semantic_type& @var{value}, const location_type& @var{location})
9187Build a complete terminal symbol which token type is @var{type}, and which
9188semantic value is @var{value}. If location tracking is enabled, also pass
9189the @var{location}.
9190@end deftypemethod
9191
9192This interface is low-level and should not be used for two reasons. First,
9193it is inconvenient, as you still have to build the semantic value, which is
9194a variant, and second, because consistency is not enforced: as with unions,
9195it is still possible to give an integer as semantic value for a string.
9196
9197So for each token type, Bison generates named constructors as follows.
9198
9199@deftypemethod {symbol_type} {} make_@var{token} (const @var{value_type}& @var{value}, const location_type& @var{location})
9200@deftypemethodx {symbol_type} {} make_@var{token} (const location_type& @var{location})
9201Build a complete terminal symbol for the token type @var{token} (not
9202including the @code{api.tokens.prefix}) whose possible semantic value is
9203@var{value} of adequate @var{value_type}. If location tracking is enabled,
9204also pass the @var{location}.
9205@end deftypemethod
9206
9207For instance, given the following declarations:
9208
9209@example
9210%define api.tokens.prefix "TOK_"
9211%token <std::string> IDENTIFIER;
9212%token <int> INTEGER;
9213%token COLON;
9214@end example
9215
9216@noindent
9217Bison generates the following functions:
9218
9219@example
9220symbol_type make_IDENTIFIER(const std::string& v,
9221 const location_type& l);
9222symbol_type make_INTEGER(const int& v,
9223 const location_type& loc);
9224symbol_type make_COLON(const location_type& loc);
9225@end example
9226
9227@noindent
9228which should be used in a Lex-scanner as follows.
9229
9230@example
9231[0-9]+ return yy::parser::make_INTEGER(text_to_int (yytext), loc);
9232[a-z]+ return yy::parser::make_IDENTIFIER(yytext, loc);
9233":" return yy::parser::make_COLON(loc);
9234@end example
9235
9236Tokens that do not have an identifier are not accessible: you cannot simply
9237use characters such as @code{':'}, they must be declared with @code{%token}.
12545799
AD
9238
9239@node A Complete C++ Example
8405b70c 9240@subsection A Complete C++ Example
12545799
AD
9241
9242This section demonstrates the use of a C++ parser with a simple but
9243complete example. This example should be available on your system,
3cdc21cf 9244ready to compile, in the directory @dfn{.../bison/examples/calc++}. It
12545799
AD
9245focuses on the use of Bison, therefore the design of the various C++
9246classes is very naive: no accessors, no encapsulation of members etc.
9247We will use a Lex scanner, and more precisely, a Flex scanner, to
3cdc21cf 9248demonstrate the various interactions. A hand-written scanner is
12545799
AD
9249actually easier to interface with.
9250
9251@menu
9252* Calc++ --- C++ Calculator:: The specifications
9253* Calc++ Parsing Driver:: An active parsing context
9254* Calc++ Parser:: A parser class
9255* Calc++ Scanner:: A pure C++ Flex scanner
9256* Calc++ Top Level:: Conducting the band
9257@end menu
9258
9259@node Calc++ --- C++ Calculator
8405b70c 9260@subsubsection Calc++ --- C++ Calculator
12545799
AD
9261
9262Of course the grammar is dedicated to arithmetics, a single
9d9b8b70 9263expression, possibly preceded by variable assignments. An
12545799
AD
9264environment containing possibly predefined variables such as
9265@code{one} and @code{two}, is exchanged with the parser. An example
9266of valid input follows.
9267
9268@example
9269three := 3
9270seven := one + two * three
9271seven * seven
9272@end example
9273
9274@node Calc++ Parsing Driver
8405b70c 9275@subsubsection Calc++ Parsing Driver
12545799
AD
9276@c - An env
9277@c - A place to store error messages
9278@c - A place for the result
9279
9280To support a pure interface with the parser (and the scanner) the
9281technique of the ``parsing context'' is convenient: a structure
9282containing all the data to exchange. Since, in addition to simply
9283launch the parsing, there are several auxiliary tasks to execute (open
9284the file for parsing, instantiate the parser etc.), we recommend
9285transforming the simple parsing context structure into a fully blown
9286@dfn{parsing driver} class.
9287
9288The declaration of this driver class, @file{calc++-driver.hh}, is as
9289follows. The first part includes the CPP guard and imports the
fb9712a9
AD
9290required standard library components, and the declaration of the parser
9291class.
12545799 9292
1c59e0a1 9293@comment file: calc++-driver.hh
12545799
AD
9294@example
9295#ifndef CALCXX_DRIVER_HH
9296# define CALCXX_DRIVER_HH
9297# include <string>
9298# include <map>
fb9712a9 9299# include "calc++-parser.hh"
12545799
AD
9300@end example
9301
12545799
AD
9302
9303@noindent
9304Then comes the declaration of the scanning function. Flex expects
9305the signature of @code{yylex} to be defined in the macro
9306@code{YY_DECL}, and the C++ parser expects it to be declared. We can
9307factor both as follows.
1c59e0a1
AD
9308
9309@comment file: calc++-driver.hh
12545799 9310@example
3dc5e96b 9311// Tell Flex the lexer's prototype ...
3cdc21cf
AD
9312# define YY_DECL \
9313 yy::calcxx_parser::symbol_type yylex (calcxx_driver& driver)
12545799
AD
9314// ... and declare it for the parser's sake.
9315YY_DECL;
9316@end example
9317
9318@noindent
9319The @code{calcxx_driver} class is then declared with its most obvious
9320members.
9321
1c59e0a1 9322@comment file: calc++-driver.hh
12545799
AD
9323@example
9324// Conducting the whole scanning and parsing of Calc++.
9325class calcxx_driver
9326@{
9327public:
9328 calcxx_driver ();
9329 virtual ~calcxx_driver ();
9330
9331 std::map<std::string, int> variables;
9332
9333 int result;
9334@end example
9335
9336@noindent
3cdc21cf
AD
9337To encapsulate the coordination with the Flex scanner, it is useful to have
9338member functions to open and close the scanning phase.
12545799 9339
1c59e0a1 9340@comment file: calc++-driver.hh
12545799
AD
9341@example
9342 // Handling the scanner.
9343 void scan_begin ();
9344 void scan_end ();
9345 bool trace_scanning;
9346@end example
9347
9348@noindent
9349Similarly for the parser itself.
9350
1c59e0a1 9351@comment file: calc++-driver.hh
12545799 9352@example
3cdc21cf
AD
9353 // Run the parser on file F.
9354 // Return 0 on success.
bb32f4f2 9355 int parse (const std::string& f);
3cdc21cf
AD
9356 // The name of the file being parsed.
9357 // Used later to pass the file name to the location tracker.
12545799 9358 std::string file;
3cdc21cf 9359 // Whether parser traces should be generated.
12545799
AD
9360 bool trace_parsing;
9361@end example
9362
9363@noindent
9364To demonstrate pure handling of parse errors, instead of simply
9365dumping them on the standard error output, we will pass them to the
9366compiler driver using the following two member functions. Finally, we
9367close the class declaration and CPP guard.
9368
1c59e0a1 9369@comment file: calc++-driver.hh
12545799
AD
9370@example
9371 // Error handling.
9372 void error (const yy::location& l, const std::string& m);
9373 void error (const std::string& m);
9374@};
9375#endif // ! CALCXX_DRIVER_HH
9376@end example
9377
9378The implementation of the driver is straightforward. The @code{parse}
9379member function deserves some attention. The @code{error} functions
9380are simple stubs, they should actually register the located error
9381messages and set error state.
9382
1c59e0a1 9383@comment file: calc++-driver.cc
12545799
AD
9384@example
9385#include "calc++-driver.hh"
9386#include "calc++-parser.hh"
9387
9388calcxx_driver::calcxx_driver ()
9389 : trace_scanning (false), trace_parsing (false)
9390@{
9391 variables["one"] = 1;
9392 variables["two"] = 2;
9393@}
9394
9395calcxx_driver::~calcxx_driver ()
9396@{
9397@}
9398
bb32f4f2 9399int
12545799
AD
9400calcxx_driver::parse (const std::string &f)
9401@{
9402 file = f;
9403 scan_begin ();
9404 yy::calcxx_parser parser (*this);
9405 parser.set_debug_level (trace_parsing);
bb32f4f2 9406 int res = parser.parse ();
12545799 9407 scan_end ();
bb32f4f2 9408 return res;
12545799
AD
9409@}
9410
9411void
9412calcxx_driver::error (const yy::location& l, const std::string& m)
9413@{
9414 std::cerr << l << ": " << m << std::endl;
9415@}
9416
9417void
9418calcxx_driver::error (const std::string& m)
9419@{
9420 std::cerr << m << std::endl;
9421@}
9422@end example
9423
9424@node Calc++ Parser
8405b70c 9425@subsubsection Calc++ Parser
12545799 9426
ff7571c0
JD
9427The grammar file @file{calc++-parser.yy} starts by asking for the C++
9428deterministic parser skeleton, the creation of the parser header file,
9429and specifies the name of the parser class. Because the C++ skeleton
9430changed several times, it is safer to require the version you designed
9431the grammar for.
1c59e0a1
AD
9432
9433@comment file: calc++-parser.yy
12545799 9434@example
ed4d67dc 9435%skeleton "lalr1.cc" /* -*- C++ -*- */
e6e704dc 9436%require "@value{VERSION}"
12545799 9437%defines
16dc6a9e 9438%define parser_class_name "calcxx_parser"
fb9712a9
AD
9439@end example
9440
3cdc21cf
AD
9441@noindent
9442@findex %define variant
9443@findex %define lex_symbol
9444This example will use genuine C++ objects as semantic values, therefore, we
9445require the variant-based interface. To make sure we properly use it, we
9446enable assertions. To fully benefit from type-safety and more natural
9447definition of ``symbol'', we enable @code{lex_symbol}.
9448
9449@comment file: calc++-parser.yy
9450@example
9451%define variant
9452%define parse.assert
9453%define lex_symbol
9454@end example
9455
fb9712a9 9456@noindent
16dc6a9e 9457@findex %code requires
3cdc21cf
AD
9458Then come the declarations/inclusions needed by the semantic values.
9459Because the parser uses the parsing driver and reciprocally, both would like
a6ca4ce2 9460to include the header of the other, which is, of course, insane. This
3cdc21cf 9461mutual dependency will be broken using forward declarations. Because the
fb9712a9 9462driver's header needs detailed knowledge about the parser class (in
3cdc21cf 9463particular its inner types), it is the parser's header which will use a
e0c07222 9464forward declaration of the driver. @xref{%code Summary}.
fb9712a9
AD
9465
9466@comment file: calc++-parser.yy
9467@example
3cdc21cf
AD
9468%code requires
9469@{
12545799 9470# include <string>
fb9712a9 9471class calcxx_driver;
9bc0dd67 9472@}
12545799
AD
9473@end example
9474
9475@noindent
9476The driver is passed by reference to the parser and to the scanner.
9477This provides a simple but effective pure interface, not relying on
9478global variables.
9479
1c59e0a1 9480@comment file: calc++-parser.yy
12545799
AD
9481@example
9482// The parsing context.
2055a44e 9483%param @{ calcxx_driver& driver @}
12545799
AD
9484@end example
9485
9486@noindent
2055a44e 9487Then we request location tracking, and initialize the
f50bfcd6 9488first location's file name. Afterward new locations are computed
12545799 9489relatively to the previous locations: the file name will be
2055a44e 9490propagated.
12545799 9491
1c59e0a1 9492@comment file: calc++-parser.yy
12545799
AD
9493@example
9494%locations
9495%initial-action
9496@{
9497 // Initialize the initial location.
b47dbebe 9498 @@$.begin.filename = @@$.end.filename = &driver.file;
12545799
AD
9499@};
9500@end example
9501
9502@noindent
2055a44e 9503Use the following two directives to enable parser tracing and verbose
12545799
AD
9504error messages.
9505
1c59e0a1 9506@comment file: calc++-parser.yy
12545799 9507@example
fa819509 9508%define parse.trace
cf499cff 9509%define parse.error verbose
12545799
AD
9510@end example
9511
fb9712a9 9512@noindent
136a0f76
PB
9513@findex %code
9514The code between @samp{%code @{} and @samp{@}} is output in the
34f98f46 9515@file{*.cc} file; it needs detailed knowledge about the driver.
fb9712a9
AD
9516
9517@comment file: calc++-parser.yy
9518@example
3cdc21cf
AD
9519%code
9520@{
fb9712a9 9521# include "calc++-driver.hh"
34f98f46 9522@}
fb9712a9
AD
9523@end example
9524
9525
12545799
AD
9526@noindent
9527The token numbered as 0 corresponds to end of file; the following line
99c08fb6 9528allows for nicer error messages referring to ``end of file'' instead of
35c1e5f0
JD
9529``$end''. Similarly user friendly names are provided for each symbol. To
9530avoid name clashes in the generated files (@pxref{Calc++ Scanner}), prefix
9531tokens with @code{TOK_} (@pxref{%define Summary,,api.tokens.prefix}).
12545799 9532
1c59e0a1 9533@comment file: calc++-parser.yy
12545799 9534@example
4c6622c2 9535%define api.tokens.prefix "TOK_"
3cdc21cf
AD
9536%token
9537 END 0 "end of file"
9538 ASSIGN ":="
9539 MINUS "-"
9540 PLUS "+"
9541 STAR "*"
9542 SLASH "/"
9543 LPAREN "("
9544 RPAREN ")"
9545;
12545799
AD
9546@end example
9547
9548@noindent
3cdc21cf
AD
9549Since we use variant-based semantic values, @code{%union} is not used, and
9550both @code{%type} and @code{%token} expect genuine types, as opposed to type
9551tags.
12545799 9552
1c59e0a1 9553@comment file: calc++-parser.yy
12545799 9554@example
3cdc21cf
AD
9555%token <std::string> IDENTIFIER "identifier"
9556%token <int> NUMBER "number"
9557%type <int> exp
9558@end example
9559
9560@noindent
9561No @code{%destructor} is needed to enable memory deallocation during error
9562recovery; the memory, for strings for instance, will be reclaimed by the
9563regular destructors. All the values are printed using their
9564@code{operator<<}.
12545799 9565
3cdc21cf
AD
9566@c FIXME: Document %printer, and mention that it takes a braced-code operand.
9567@comment file: calc++-parser.yy
9568@example
9569%printer @{ debug_stream () << $$; @} <*>;
12545799
AD
9570@end example
9571
9572@noindent
3cdc21cf
AD
9573The grammar itself is straightforward (@pxref{Location Tracking Calc, ,
9574Location Tracking Calculator: @code{ltcalc}}).
12545799 9575
1c59e0a1 9576@comment file: calc++-parser.yy
12545799
AD
9577@example
9578%%
9579%start unit;
9580unit: assignments exp @{ driver.result = $2; @};
9581
99c08fb6
AD
9582assignments:
9583 assignments assignment @{@}
9584| /* Nothing. */ @{@};
12545799 9585
3dc5e96b 9586assignment:
3cdc21cf 9587 "identifier" ":=" exp @{ driver.variables[$1] = $3; @};
12545799 9588
3cdc21cf
AD
9589%left "+" "-";
9590%left "*" "/";
99c08fb6 9591exp:
3cdc21cf
AD
9592 exp "+" exp @{ $$ = $1 + $3; @}
9593| exp "-" exp @{ $$ = $1 - $3; @}
9594| exp "*" exp @{ $$ = $1 * $3; @}
9595| exp "/" exp @{ $$ = $1 / $3; @}
298e8ad9 9596| "(" exp ")" @{ std::swap ($$, $2); @}
3cdc21cf 9597| "identifier" @{ $$ = driver.variables[$1]; @}
298e8ad9 9598| "number" @{ std::swap ($$, $1); @};
12545799
AD
9599%%
9600@end example
9601
9602@noindent
9603Finally the @code{error} member function registers the errors to the
9604driver.
9605
1c59e0a1 9606@comment file: calc++-parser.yy
12545799
AD
9607@example
9608void
3cdc21cf 9609yy::calcxx_parser::error (const location_type& l,
1c59e0a1 9610 const std::string& m)
12545799
AD
9611@{
9612 driver.error (l, m);
9613@}
9614@end example
9615
9616@node Calc++ Scanner
8405b70c 9617@subsubsection Calc++ Scanner
12545799
AD
9618
9619The Flex scanner first includes the driver declaration, then the
9620parser's to get the set of defined tokens.
9621
1c59e0a1 9622@comment file: calc++-scanner.ll
12545799
AD
9623@example
9624%@{ /* -*- C++ -*- */
3c248d70
AD
9625# include <cerrno>
9626# include <climits>
3cdc21cf 9627# include <cstdlib>
12545799
AD
9628# include <string>
9629# include "calc++-driver.hh"
9630# include "calc++-parser.hh"
eaea13f5 9631
3cdc21cf
AD
9632// Work around an incompatibility in flex (at least versions
9633// 2.5.31 through 2.5.33): it generates code that does
9634// not conform to C89. See Debian bug 333231
9635// <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>.
7870f699
PE
9636# undef yywrap
9637# define yywrap() 1
eaea13f5 9638
3cdc21cf
AD
9639// The location of the current token.
9640static yy::location loc;
12545799
AD
9641%@}
9642@end example
9643
9644@noindent
9645Because there is no @code{#include}-like feature we don't need
9646@code{yywrap}, we don't need @code{unput} either, and we parse an
9647actual file, this is not an interactive session with the user.
3cdc21cf 9648Finally, we enable scanner tracing.
12545799 9649
1c59e0a1 9650@comment file: calc++-scanner.ll
12545799
AD
9651@example
9652%option noyywrap nounput batch debug
9653@end example
9654
9655@noindent
9656Abbreviations allow for more readable rules.
9657
1c59e0a1 9658@comment file: calc++-scanner.ll
12545799
AD
9659@example
9660id [a-zA-Z][a-zA-Z_0-9]*
9661int [0-9]+
9662blank [ \t]
9663@end example
9664
9665@noindent
9d9b8b70 9666The following paragraph suffices to track locations accurately. Each
12545799 9667time @code{yylex} is invoked, the begin position is moved onto the end
3cdc21cf
AD
9668position. Then when a pattern is matched, its width is added to the end
9669column. When matching ends of lines, the end
12545799
AD
9670cursor is adjusted, and each time blanks are matched, the begin cursor
9671is moved onto the end cursor to effectively ignore the blanks
9672preceding tokens. Comments would be treated equally.
9673
1c59e0a1 9674@comment file: calc++-scanner.ll
12545799 9675@example
828c373b 9676%@{
3cdc21cf
AD
9677 // Code run each time a pattern is matched.
9678 # define YY_USER_ACTION loc.columns (yyleng);
828c373b 9679%@}
12545799
AD
9680%%
9681%@{
3cdc21cf
AD
9682 // Code run each time yylex is called.
9683 loc.step ();
12545799 9684%@}
3cdc21cf
AD
9685@{blank@}+ loc.step ();
9686[\n]+ loc.lines (yyleng); loc.step ();
12545799
AD
9687@end example
9688
9689@noindent
3cdc21cf 9690The rules are simple. The driver is used to report errors.
12545799 9691
1c59e0a1 9692@comment file: calc++-scanner.ll
12545799 9693@example
3cdc21cf
AD
9694"-" return yy::calcxx_parser::make_MINUS(loc);
9695"+" return yy::calcxx_parser::make_PLUS(loc);
9696"*" return yy::calcxx_parser::make_STAR(loc);
9697"/" return yy::calcxx_parser::make_SLASH(loc);
9698"(" return yy::calcxx_parser::make_LPAREN(loc);
9699")" return yy::calcxx_parser::make_RPAREN(loc);
9700":=" return yy::calcxx_parser::make_ASSIGN(loc);
9701
04098407
PE
9702@{int@} @{
9703 errno = 0;
9704 long n = strtol (yytext, NULL, 10);
9705 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
3cdc21cf
AD
9706 driver.error (loc, "integer is out of range");
9707 return yy::calcxx_parser::make_NUMBER(n, loc);
04098407 9708@}
3cdc21cf
AD
9709@{id@} return yy::calcxx_parser::make_IDENTIFIER(yytext, loc);
9710. driver.error (loc, "invalid character");
9711<<EOF>> return yy::calcxx_parser::make_END(loc);
12545799
AD
9712%%
9713@end example
9714
9715@noindent
3cdc21cf 9716Finally, because the scanner-related driver's member-functions depend
12545799
AD
9717on the scanner's data, it is simpler to implement them in this file.
9718
1c59e0a1 9719@comment file: calc++-scanner.ll
12545799
AD
9720@example
9721void
9722calcxx_driver::scan_begin ()
9723@{
9724 yy_flex_debug = trace_scanning;
bb32f4f2
AD
9725 if (file == "-")
9726 yyin = stdin;
9727 else if (!(yyin = fopen (file.c_str (), "r")))
9728 @{
3cdc21cf 9729 error (std::string ("cannot open ") + file + ": " + strerror(errno));
bb32f4f2
AD
9730 exit (1);
9731 @}
12545799
AD
9732@}
9733
9734void
9735calcxx_driver::scan_end ()
9736@{
9737 fclose (yyin);
9738@}
9739@end example
9740
9741@node Calc++ Top Level
8405b70c 9742@subsubsection Calc++ Top Level
12545799
AD
9743
9744The top level file, @file{calc++.cc}, poses no problem.
9745
1c59e0a1 9746@comment file: calc++.cc
12545799
AD
9747@example
9748#include <iostream>
9749#include "calc++-driver.hh"
9750
9751int
fa4d969f 9752main (int argc, char *argv[])
12545799 9753@{
414c76a4 9754 int res = 0;
12545799
AD
9755 calcxx_driver driver;
9756 for (++argv; argv[0]; ++argv)
9757 if (*argv == std::string ("-p"))
9758 driver.trace_parsing = true;
9759 else if (*argv == std::string ("-s"))
9760 driver.trace_scanning = true;
bb32f4f2
AD
9761 else if (!driver.parse (*argv))
9762 std::cout << driver.result << std::endl;
414c76a4
AD
9763 else
9764 res = 1;
9765 return res;
12545799
AD
9766@}
9767@end example
9768
8405b70c
PB
9769@node Java Parsers
9770@section Java Parsers
9771
9772@menu
f5f419de
DJ
9773* Java Bison Interface:: Asking for Java parser generation
9774* Java Semantic Values:: %type and %token vs. Java
9775* Java Location Values:: The position and location classes
9776* Java Parser Interface:: Instantiating and running the parser
9777* Java Scanner Interface:: Specifying the scanner for the parser
9778* Java Action Features:: Special features for use in actions
9779* Java Differences:: Differences between C/C++ and Java Grammars
9780* Java Declarations Summary:: List of Bison declarations used with Java
8405b70c
PB
9781@end menu
9782
9783@node Java Bison Interface
9784@subsection Java Bison Interface
9785@c - %language "Java"
8405b70c 9786
59da312b
JD
9787(The current Java interface is experimental and may evolve.
9788More user feedback will help to stabilize it.)
9789
e254a580
DJ
9790The Java parser skeletons are selected using the @code{%language "Java"}
9791directive or the @option{-L java}/@option{--language=java} option.
8405b70c 9792
e254a580 9793@c FIXME: Documented bug.
ff7571c0
JD
9794When generating a Java parser, @code{bison @var{basename}.y} will
9795create a single Java source file named @file{@var{basename}.java}
9796containing the parser implementation. Using a grammar file without a
9797@file{.y} suffix is currently broken. The basename of the parser
9798implementation file can be changed by the @code{%file-prefix}
9799directive or the @option{-p}/@option{--name-prefix} option. The
9800entire parser implementation file name can be changed by the
9801@code{%output} directive or the @option{-o}/@option{--output} option.
9802The parser implementation file contains a single class for the parser.
8405b70c 9803
e254a580 9804You can create documentation for generated parsers using Javadoc.
8405b70c 9805
e254a580
DJ
9806Contrary to C parsers, Java parsers do not use global variables; the
9807state of the parser is always local to an instance of the parser class.
9808Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
67501061 9809and @samp{%define api.pure} directives does not do anything when used in
e254a580 9810Java.
8405b70c 9811
e254a580 9812Push parsers are currently unsupported in Java and @code{%define
67212941 9813api.push-pull} have no effect.
01b477c6 9814
8a4281b9 9815GLR parsers are currently unsupported in Java. Do not use the
e254a580
DJ
9816@code{glr-parser} directive.
9817
9818No header file can be generated for Java parsers. Do not use the
9819@code{%defines} directive or the @option{-d}/@option{--defines} options.
9820
9821@c FIXME: Possible code change.
fa819509
AD
9822Currently, support for tracing is always compiled
9823in. Thus the @samp{%define parse.trace} and @samp{%token-table}
9824directives and the
e254a580
DJ
9825@option{-t}/@option{--debug} and @option{-k}/@option{--token-table}
9826options have no effect. This may change in the future to eliminate
fa819509
AD
9827unused code in the generated parser, so use @samp{%define parse.trace}
9828explicitly
1979121c 9829if needed. Also, in the future the
e254a580
DJ
9830@code{%token-table} directive might enable a public interface to
9831access the token names and codes.
8405b70c 9832
09ccae9b 9833Getting a ``code too large'' error from the Java compiler means the code
f50bfcd6 9834hit the 64KB bytecode per method limitation of the Java class file.
09ccae9b
DJ
9835Try reducing the amount of code in actions and static initializers;
9836otherwise, report a bug so that the parser skeleton will be improved.
9837
9838
8405b70c
PB
9839@node Java Semantic Values
9840@subsection Java Semantic Values
9841@c - No %union, specify type in %type/%token.
9842@c - YYSTYPE
9843@c - Printer and destructor
9844
9845There is no @code{%union} directive in Java parsers. Instead, the
9846semantic values' types (class names) should be specified in the
9847@code{%type} or @code{%token} directive:
9848
9849@example
9850%type <Expression> expr assignment_expr term factor
9851%type <Integer> number
9852@end example
9853
9854By default, the semantic stack is declared to have @code{Object} members,
9855which means that the class types you specify can be of any class.
9856To improve the type safety of the parser, you can declare the common
67501061 9857superclass of all the semantic values using the @samp{%define stype}
e254a580 9858directive. For example, after the following declaration:
8405b70c
PB
9859
9860@example
e254a580 9861%define stype "ASTNode"
8405b70c
PB
9862@end example
9863
9864@noindent
9865any @code{%type} or @code{%token} specifying a semantic type which
9866is not a subclass of ASTNode, will cause a compile-time error.
9867
e254a580 9868@c FIXME: Documented bug.
8405b70c
PB
9869Types used in the directives may be qualified with a package name.
9870Primitive data types are accepted for Java version 1.5 or later. Note
9871that in this case the autoboxing feature of Java 1.5 will be used.
e254a580
DJ
9872Generic types may not be used; this is due to a limitation in the
9873implementation of Bison, and may change in future releases.
8405b70c
PB
9874
9875Java parsers do not support @code{%destructor}, since the language
9876adopts garbage collection. The parser will try to hold references
9877to semantic values for as little time as needed.
9878
9879Java parsers do not support @code{%printer}, as @code{toString()}
9880can be used to print the semantic values. This however may change
9881(in a backwards-compatible way) in future versions of Bison.
9882
9883
9884@node Java Location Values
9885@subsection Java Location Values
9886@c - %locations
9887@c - class Position
9888@c - class Location
9889
9890When the directive @code{%locations} is used, the Java parser
9891supports location tracking, see @ref{Locations, , Locations Overview}.
9892An auxiliary user-defined class defines a @dfn{position}, a single point
9893in a file; Bison itself defines a class representing a @dfn{location},
9894a range composed of a pair of positions (possibly spanning several
9895files). The location class is an inner class of the parser; the name
e254a580 9896is @code{Location} by default, and may also be renamed using
cf499cff 9897@samp{%define location_type "@var{class-name}"}.
8405b70c
PB
9898
9899The location class treats the position as a completely opaque value.
9900By default, the class name is @code{Position}, but this can be changed
67501061 9901with @samp{%define position_type "@var{class-name}"}. This class must
e254a580 9902be supplied by the user.
8405b70c
PB
9903
9904
e254a580
DJ
9905@deftypeivar {Location} {Position} begin
9906@deftypeivarx {Location} {Position} end
8405b70c 9907The first, inclusive, position of the range, and the first beyond.
e254a580
DJ
9908@end deftypeivar
9909
9910@deftypeop {Constructor} {Location} {} Location (Position @var{loc})
c265fd6b 9911Create a @code{Location} denoting an empty range located at a given point.
e254a580 9912@end deftypeop
8405b70c 9913
e254a580
DJ
9914@deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end})
9915Create a @code{Location} from the endpoints of the range.
9916@end deftypeop
9917
9918@deftypemethod {Location} {String} toString ()
8405b70c
PB
9919Prints the range represented by the location. For this to work
9920properly, the position class should override the @code{equals} and
9921@code{toString} methods appropriately.
9922@end deftypemethod
9923
9924
9925@node Java Parser Interface
9926@subsection Java Parser Interface
9927@c - define parser_class_name
9928@c - Ctor
9929@c - parse, error, set_debug_level, debug_level, set_debug_stream,
9930@c debug_stream.
9931@c - Reporting errors
9932
e254a580
DJ
9933The name of the generated parser class defaults to @code{YYParser}. The
9934@code{YY} prefix may be changed using the @code{%name-prefix} directive
9935or the @option{-p}/@option{--name-prefix} option. Alternatively, use
67501061 9936@samp{%define parser_class_name "@var{name}"} to give a custom name to
e254a580 9937the class. The interface of this class is detailed below.
8405b70c 9938
e254a580 9939By default, the parser class has package visibility. A declaration
67501061 9940@samp{%define public} will change to public visibility. Remember that,
e254a580
DJ
9941according to the Java language specification, the name of the @file{.java}
9942file should match the name of the class in this case. Similarly, you can
9943use @code{abstract}, @code{final} and @code{strictfp} with the
9944@code{%define} declaration to add other modifiers to the parser class.
67501061 9945A single @samp{%define annotations "@var{annotations}"} directive can
1979121c 9946be used to add any number of annotations to the parser class.
e254a580
DJ
9947
9948The Java package name of the parser class can be specified using the
67501061 9949@samp{%define package} directive. The superclass and the implemented
e254a580 9950interfaces of the parser class can be specified with the @code{%define
67501061 9951extends} and @samp{%define implements} directives.
e254a580
DJ
9952
9953The parser class defines an inner class, @code{Location}, that is used
9954for location tracking (see @ref{Java Location Values}), and a inner
9955interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
9956these inner class/interface, and the members described in the interface
9957below, all the other members and fields are preceded with a @code{yy} or
9958@code{YY} prefix to avoid clashes with user code.
9959
e254a580
DJ
9960The parser class can be extended using the @code{%parse-param}
9961directive. Each occurrence of the directive will add a @code{protected
9962final} field to the parser class, and an argument to its constructor,
9963which initialize them automatically.
9964
e254a580
DJ
9965@deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
9966Build a new parser object with embedded @code{%code lexer}. There are
2055a44e
AD
9967no parameters, unless @code{%param}s and/or @code{%parse-param}s and/or
9968@code{%lex-param}s are used.
1979121c
DJ
9969
9970Use @code{%code init} for code added to the start of the constructor
9971body. This is especially useful to initialize superclasses. Use
f50bfcd6 9972@samp{%define init_throws} to specify any uncaught exceptions.
e254a580
DJ
9973@end deftypeop
9974
9975@deftypeop {Constructor} {YYParser} {} YYParser (Lexer @var{lexer}, @var{parse_param}, @dots{})
9976Build a new parser object using the specified scanner. There are no
2055a44e
AD
9977additional parameters unless @code{%param}s and/or @code{%parse-param}s are
9978used.
e254a580
DJ
9979
9980If the scanner is defined by @code{%code lexer}, this constructor is
9981declared @code{protected} and is called automatically with a scanner
2055a44e 9982created with the correct @code{%param}s and/or @code{%lex-param}s.
1979121c
DJ
9983
9984Use @code{%code init} for code added to the start of the constructor
9985body. This is especially useful to initialize superclasses. Use
67501061 9986@samp{%define init_throws} to specify any uncatch exceptions.
e254a580 9987@end deftypeop
8405b70c
PB
9988
9989@deftypemethod {YYParser} {boolean} parse ()
9990Run the syntactic analysis, and return @code{true} on success,
9991@code{false} otherwise.
9992@end deftypemethod
9993
1979121c
DJ
9994@deftypemethod {YYParser} {boolean} getErrorVerbose ()
9995@deftypemethodx {YYParser} {void} setErrorVerbose (boolean @var{verbose})
9996Get or set the option to produce verbose error messages. These are only
cf499cff 9997available with @samp{%define parse.error verbose}, which also turns on
1979121c
DJ
9998verbose error messages.
9999@end deftypemethod
10000
10001@deftypemethod {YYParser} {void} yyerror (String @var{msg})
10002@deftypemethodx {YYParser} {void} yyerror (Position @var{pos}, String @var{msg})
10003@deftypemethodx {YYParser} {void} yyerror (Location @var{loc}, String @var{msg})
10004Print an error message using the @code{yyerror} method of the scanner
10005instance in use. The @code{Location} and @code{Position} parameters are
10006available only if location tracking is active.
10007@end deftypemethod
10008
01b477c6 10009@deftypemethod {YYParser} {boolean} recovering ()
8405b70c 10010During the syntactic analysis, return @code{true} if recovering
e254a580
DJ
10011from a syntax error.
10012@xref{Error Recovery}.
8405b70c
PB
10013@end deftypemethod
10014
10015@deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
10016@deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
10017Get or set the stream used for tracing the parsing. It defaults to
10018@code{System.err}.
10019@end deftypemethod
10020
10021@deftypemethod {YYParser} {int} getDebugLevel ()
10022@deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
10023Get or set the tracing level. Currently its value is either 0, no trace,
10024or nonzero, full tracing.
10025@end deftypemethod
10026
1979121c
DJ
10027@deftypecv {Constant} {YYParser} {String} {bisonVersion}
10028@deftypecvx {Constant} {YYParser} {String} {bisonSkeleton}
10029Identify the Bison version and skeleton used to generate this parser.
10030@end deftypecv
10031
8405b70c
PB
10032
10033@node Java Scanner Interface
10034@subsection Java Scanner Interface
01b477c6 10035@c - %code lexer
8405b70c 10036@c - %lex-param
01b477c6 10037@c - Lexer interface
8405b70c 10038
e254a580
DJ
10039There are two possible ways to interface a Bison-generated Java parser
10040with a scanner: the scanner may be defined by @code{%code lexer}, or
10041defined elsewhere. In either case, the scanner has to implement the
1979121c
DJ
10042@code{Lexer} inner interface of the parser class. This interface also
10043contain constants for all user-defined token names and the predefined
10044@code{EOF} token.
e254a580
DJ
10045
10046In the first case, the body of the scanner class is placed in
10047@code{%code lexer} blocks. If you want to pass parameters from the
10048parser constructor to the scanner constructor, specify them with
10049@code{%lex-param}; they are passed before @code{%parse-param}s to the
10050constructor.
01b477c6 10051
59c5ac72 10052In the second case, the scanner has to implement the @code{Lexer} interface,
01b477c6
PB
10053which is defined within the parser class (e.g., @code{YYParser.Lexer}).
10054The constructor of the parser object will then accept an object
10055implementing the interface; @code{%lex-param} is not used in this
10056case.
10057
10058In both cases, the scanner has to implement the following methods.
10059
e254a580
DJ
10060@deftypemethod {Lexer} {void} yyerror (Location @var{loc}, String @var{msg})
10061This method is defined by the user to emit an error message. The first
10062parameter is omitted if location tracking is not active. Its type can be
67501061 10063changed using @samp{%define location_type "@var{class-name}".}
8405b70c
PB
10064@end deftypemethod
10065
e254a580 10066@deftypemethod {Lexer} {int} yylex ()
8405b70c 10067Return the next token. Its type is the return value, its semantic
f50bfcd6 10068value and location are saved and returned by the their methods in the
e254a580
DJ
10069interface.
10070
67501061 10071Use @samp{%define lex_throws} to specify any uncaught exceptions.
e254a580 10072Default is @code{java.io.IOException}.
8405b70c
PB
10073@end deftypemethod
10074
10075@deftypemethod {Lexer} {Position} getStartPos ()
10076@deftypemethodx {Lexer} {Position} getEndPos ()
01b477c6
PB
10077Return respectively the first position of the last token that
10078@code{yylex} returned, and the first position beyond it. These
10079methods are not needed unless location tracking is active.
8405b70c 10080
67501061 10081The return type can be changed using @samp{%define position_type
8405b70c
PB
10082"@var{class-name}".}
10083@end deftypemethod
10084
10085@deftypemethod {Lexer} {Object} getLVal ()
f50bfcd6 10086Return the semantic value of the last token that yylex returned.
8405b70c 10087
67501061 10088The return type can be changed using @samp{%define stype
8405b70c
PB
10089"@var{class-name}".}
10090@end deftypemethod
10091
10092
e254a580
DJ
10093@node Java Action Features
10094@subsection Special Features for Use in Java Actions
10095
10096The following special constructs can be uses in Java actions.
10097Other analogous C action features are currently unavailable for Java.
10098
67501061 10099Use @samp{%define throws} to specify any uncaught exceptions from parser
e254a580
DJ
10100actions, and initial actions specified by @code{%initial-action}.
10101
10102@defvar $@var{n}
10103The semantic value for the @var{n}th component of the current rule.
10104This may not be assigned to.
10105@xref{Java Semantic Values}.
10106@end defvar
10107
10108@defvar $<@var{typealt}>@var{n}
10109Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
10110@xref{Java Semantic Values}.
10111@end defvar
10112
10113@defvar $$
10114The semantic value for the grouping made by the current rule. As a
10115value, this is in the base type (@code{Object} or as specified by
67501061 10116@samp{%define stype}) as in not cast to the declared subtype because
e254a580
DJ
10117casts are not allowed on the left-hand side of Java assignments.
10118Use an explicit Java cast if the correct subtype is needed.
10119@xref{Java Semantic Values}.
10120@end defvar
10121
10122@defvar $<@var{typealt}>$
10123Same as @code{$$} since Java always allow assigning to the base type.
10124Perhaps we should use this and @code{$<>$} for the value and @code{$$}
10125for setting the value but there is currently no easy way to distinguish
10126these constructs.
10127@xref{Java Semantic Values}.
10128@end defvar
10129
10130@defvar @@@var{n}
10131The location information of the @var{n}th component of the current rule.
10132This may not be assigned to.
10133@xref{Java Location Values}.
10134@end defvar
10135
10136@defvar @@$
10137The location information of the grouping made by the current rule.
10138@xref{Java Location Values}.
10139@end defvar
10140
10141@deffn {Statement} {return YYABORT;}
10142Return immediately from the parser, indicating failure.
10143@xref{Java Parser Interface}.
10144@end deffn
8405b70c 10145
e254a580
DJ
10146@deffn {Statement} {return YYACCEPT;}
10147Return immediately from the parser, indicating success.
10148@xref{Java Parser Interface}.
10149@end deffn
8405b70c 10150
e254a580 10151@deffn {Statement} {return YYERROR;}
c265fd6b 10152Start error recovery without printing an error message.
e254a580
DJ
10153@xref{Error Recovery}.
10154@end deffn
8405b70c 10155
e254a580
DJ
10156@deftypefn {Function} {boolean} recovering ()
10157Return whether error recovery is being done. In this state, the parser
10158reads token until it reaches a known state, and then restarts normal
10159operation.
10160@xref{Error Recovery}.
10161@end deftypefn
8405b70c 10162
1979121c
DJ
10163@deftypefn {Function} {void} yyerror (String @var{msg})
10164@deftypefnx {Function} {void} yyerror (Position @var{loc}, String @var{msg})
10165@deftypefnx {Function} {void} yyerror (Location @var{loc}, String @var{msg})
e254a580 10166Print an error message using the @code{yyerror} method of the scanner
1979121c
DJ
10167instance in use. The @code{Location} and @code{Position} parameters are
10168available only if location tracking is active.
e254a580 10169@end deftypefn
8405b70c 10170
8405b70c 10171
8405b70c
PB
10172@node Java Differences
10173@subsection Differences between C/C++ and Java Grammars
10174
10175The different structure of the Java language forces several differences
10176between C/C++ grammars, and grammars designed for Java parsers. This
29553547 10177section summarizes these differences.
8405b70c
PB
10178
10179@itemize
10180@item
01b477c6 10181Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
8405b70c 10182@code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
01b477c6
PB
10183macros. Instead, they should be preceded by @code{return} when they
10184appear in an action. The actual definition of these symbols is
8405b70c
PB
10185opaque to the Bison grammar, and it might change in the future. The
10186only meaningful operation that you can do, is to return them.
e254a580 10187See @pxref{Java Action Features}.
8405b70c
PB
10188
10189Note that of these three symbols, only @code{YYACCEPT} and
10190@code{YYABORT} will cause a return from the @code{yyparse}
10191method@footnote{Java parsers include the actions in a separate
10192method than @code{yyparse} in order to have an intuitive syntax that
10193corresponds to these C macros.}.
10194
e254a580
DJ
10195@item
10196Java lacks unions, so @code{%union} has no effect. Instead, semantic
10197values have a common base type: @code{Object} or as specified by
f50bfcd6 10198@samp{%define stype}. Angle brackets on @code{%token}, @code{type},
e254a580
DJ
10199@code{$@var{n}} and @code{$$} specify subtypes rather than fields of
10200an union. The type of @code{$$}, even with angle brackets, is the base
10201type since Java casts are not allow on the left-hand side of assignments.
10202Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
10203left-hand side of assignments. See @pxref{Java Semantic Values} and
10204@pxref{Java Action Features}.
10205
8405b70c 10206@item
f50bfcd6 10207The prologue declarations have a different meaning than in C/C++ code.
01b477c6
PB
10208@table @asis
10209@item @code{%code imports}
10210blocks are placed at the beginning of the Java source code. They may
10211include copyright notices. For a @code{package} declarations, it is
67501061 10212suggested to use @samp{%define package} instead.
8405b70c 10213
01b477c6
PB
10214@item unqualified @code{%code}
10215blocks are placed inside the parser class.
10216
10217@item @code{%code lexer}
10218blocks, if specified, should include the implementation of the
10219scanner. If there is no such block, the scanner can be any class
10220that implements the appropriate interface (see @pxref{Java Scanner
10221Interface}).
29553547 10222@end table
8405b70c
PB
10223
10224Other @code{%code} blocks are not supported in Java parsers.
e254a580
DJ
10225In particular, @code{%@{ @dots{} %@}} blocks should not be used
10226and may give an error in future versions of Bison.
10227
01b477c6 10228The epilogue has the same meaning as in C/C++ code and it can
e254a580
DJ
10229be used to define other classes used by the parser @emph{outside}
10230the parser class.
8405b70c
PB
10231@end itemize
10232
e254a580
DJ
10233
10234@node Java Declarations Summary
10235@subsection Java Declarations Summary
10236
10237This summary only include declarations specific to Java or have special
10238meaning when used in a Java parser.
10239
10240@deffn {Directive} {%language "Java"}
10241Generate a Java class for the parser.
10242@end deffn
10243
10244@deffn {Directive} %lex-param @{@var{type} @var{name}@}
10245A parameter for the lexer class defined by @code{%code lexer}
10246@emph{only}, added as parameters to the lexer constructor and the parser
10247constructor that @emph{creates} a lexer. Default is none.
10248@xref{Java Scanner Interface}.
10249@end deffn
10250
10251@deffn {Directive} %name-prefix "@var{prefix}"
10252The prefix of the parser class name @code{@var{prefix}Parser} if
67501061 10253@samp{%define parser_class_name} is not used. Default is @code{YY}.
e254a580
DJ
10254@xref{Java Bison Interface}.
10255@end deffn
10256
10257@deffn {Directive} %parse-param @{@var{type} @var{name}@}
10258A parameter for the parser class added as parameters to constructor(s)
10259and as fields initialized by the constructor(s). Default is none.
10260@xref{Java Parser Interface}.
10261@end deffn
10262
10263@deffn {Directive} %token <@var{type}> @var{token} @dots{}
10264Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
10265@xref{Java Semantic Values}.
10266@end deffn
10267
10268@deffn {Directive} %type <@var{type}> @var{nonterminal} @dots{}
10269Declare the type of nonterminals. Note that the angle brackets enclose
10270a Java @emph{type}.
10271@xref{Java Semantic Values}.
10272@end deffn
10273
10274@deffn {Directive} %code @{ @var{code} @dots{} @}
10275Code appended to the inside of the parser class.
10276@xref{Java Differences}.
10277@end deffn
10278
10279@deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
10280Code inserted just after the @code{package} declaration.
10281@xref{Java Differences}.
10282@end deffn
10283
1979121c
DJ
10284@deffn {Directive} {%code init} @{ @var{code} @dots{} @}
10285Code inserted at the beginning of the parser constructor body.
10286@xref{Java Parser Interface}.
10287@end deffn
10288
e254a580
DJ
10289@deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
10290Code added to the body of a inner lexer class within the parser class.
10291@xref{Java Scanner Interface}.
10292@end deffn
10293
10294@deffn {Directive} %% @var{code} @dots{}
10295Code (after the second @code{%%}) appended to the end of the file,
10296@emph{outside} the parser class.
10297@xref{Java Differences}.
10298@end deffn
10299
10300@deffn {Directive} %@{ @var{code} @dots{} %@}
1979121c 10301Not supported. Use @code{%code imports} instead.
e254a580
DJ
10302@xref{Java Differences}.
10303@end deffn
10304
10305@deffn {Directive} {%define abstract}
10306Whether the parser class is declared @code{abstract}. Default is false.
10307@xref{Java Bison Interface}.
10308@end deffn
10309
1979121c
DJ
10310@deffn {Directive} {%define annotations} "@var{annotations}"
10311The Java annotations for the parser class. Default is none.
10312@xref{Java Bison Interface}.
10313@end deffn
10314
e254a580
DJ
10315@deffn {Directive} {%define extends} "@var{superclass}"
10316The superclass of the parser class. Default is none.
10317@xref{Java Bison Interface}.
10318@end deffn
10319
10320@deffn {Directive} {%define final}
10321Whether the parser class is declared @code{final}. Default is false.
10322@xref{Java Bison Interface}.
10323@end deffn
10324
10325@deffn {Directive} {%define implements} "@var{interfaces}"
10326The implemented interfaces of the parser class, a comma-separated list.
10327Default is none.
10328@xref{Java Bison Interface}.
10329@end deffn
10330
1979121c
DJ
10331@deffn {Directive} {%define init_throws} "@var{exceptions}"
10332The exceptions thrown by @code{%code init} from the parser class
10333constructor. Default is none.
10334@xref{Java Parser Interface}.
10335@end deffn
10336
e254a580
DJ
10337@deffn {Directive} {%define lex_throws} "@var{exceptions}"
10338The exceptions thrown by the @code{yylex} method of the lexer, a
10339comma-separated list. Default is @code{java.io.IOException}.
10340@xref{Java Scanner Interface}.
10341@end deffn
10342
10343@deffn {Directive} {%define location_type} "@var{class}"
10344The name of the class used for locations (a range between two
10345positions). This class is generated as an inner class of the parser
10346class by @command{bison}. Default is @code{Location}.
10347@xref{Java Location Values}.
10348@end deffn
10349
10350@deffn {Directive} {%define package} "@var{package}"
10351The package to put the parser class in. Default is none.
10352@xref{Java Bison Interface}.
10353@end deffn
10354
10355@deffn {Directive} {%define parser_class_name} "@var{name}"
10356The name of the parser class. Default is @code{YYParser} or
10357@code{@var{name-prefix}Parser}.
10358@xref{Java Bison Interface}.
10359@end deffn
10360
10361@deffn {Directive} {%define position_type} "@var{class}"
10362The name of the class used for positions. This class must be supplied by
10363the user. Default is @code{Position}.
10364@xref{Java Location Values}.
10365@end deffn
10366
10367@deffn {Directive} {%define public}
10368Whether the parser class is declared @code{public}. Default is false.
10369@xref{Java Bison Interface}.
10370@end deffn
10371
10372@deffn {Directive} {%define stype} "@var{class}"
10373The base type of semantic values. Default is @code{Object}.
10374@xref{Java Semantic Values}.
10375@end deffn
10376
10377@deffn {Directive} {%define strictfp}
10378Whether the parser class is declared @code{strictfp}. Default is false.
10379@xref{Java Bison Interface}.
10380@end deffn
10381
10382@deffn {Directive} {%define throws} "@var{exceptions}"
10383The exceptions thrown by user-supplied parser actions and
10384@code{%initial-action}, a comma-separated list. Default is none.
10385@xref{Java Parser Interface}.
10386@end deffn
10387
10388
12545799 10389@c ================================================= FAQ
d1a1114f
AD
10390
10391@node FAQ
10392@chapter Frequently Asked Questions
10393@cindex frequently asked questions
10394@cindex questions
10395
10396Several questions about Bison come up occasionally. Here some of them
10397are addressed.
10398
10399@menu
55ba27be
AD
10400* Memory Exhausted:: Breaking the Stack Limits
10401* How Can I Reset the Parser:: @code{yyparse} Keeps some State
10402* Strings are Destroyed:: @code{yylval} Loses Track of Strings
10403* Implementing Gotos/Loops:: Control Flow in the Calculator
ed2e6384 10404* Multiple start-symbols:: Factoring closely related grammars
8a4281b9 10405* Secure? Conform?:: Is Bison POSIX safe?
55ba27be
AD
10406* I can't build Bison:: Troubleshooting
10407* Where can I find help?:: Troubleshouting
10408* Bug Reports:: Troublereporting
8405b70c 10409* More Languages:: Parsers in C++, Java, and so on
55ba27be
AD
10410* Beta Testing:: Experimenting development versions
10411* Mailing Lists:: Meeting other Bison users
d1a1114f
AD
10412@end menu
10413
1a059451
PE
10414@node Memory Exhausted
10415@section Memory Exhausted
d1a1114f
AD
10416
10417@display
1a059451 10418My parser returns with error with a @samp{memory exhausted}
d1a1114f
AD
10419message. What can I do?
10420@end display
10421
10422This question is already addressed elsewhere, @xref{Recursion,
10423,Recursive Rules}.
10424
e64fec0a
PE
10425@node How Can I Reset the Parser
10426@section How Can I Reset the Parser
5b066063 10427
0e14ad77
PE
10428The following phenomenon has several symptoms, resulting in the
10429following typical questions:
5b066063
AD
10430
10431@display
10432I invoke @code{yyparse} several times, and on correct input it works
10433properly; but when a parse error is found, all the other calls fail
0e14ad77 10434too. How can I reset the error flag of @code{yyparse}?
5b066063
AD
10435@end display
10436
10437@noindent
10438or
10439
10440@display
0e14ad77 10441My parser includes support for an @samp{#include}-like feature, in
5b066063 10442which case I run @code{yyparse} from @code{yyparse}. This fails
67501061 10443although I did specify @samp{%define api.pure}.
5b066063
AD
10444@end display
10445
0e14ad77
PE
10446These problems typically come not from Bison itself, but from
10447Lex-generated scanners. Because these scanners use large buffers for
5b066063
AD
10448speed, they might not notice a change of input file. As a
10449demonstration, consider the following source file,
10450@file{first-line.l}:
10451
10452@verbatim
10453%{
10454#include <stdio.h>
10455#include <stdlib.h>
10456%}
10457%%
10458.*\n ECHO; return 1;
10459%%
10460int
0e14ad77 10461yyparse (char const *file)
5b066063
AD
10462{
10463 yyin = fopen (file, "r");
10464 if (!yyin)
10465 exit (2);
fa7e68c3 10466 /* One token only. */
5b066063 10467 yylex ();
0e14ad77 10468 if (fclose (yyin) != 0)
5b066063
AD
10469 exit (3);
10470 return 0;
10471}
10472
10473int
0e14ad77 10474main (void)
5b066063
AD
10475{
10476 yyparse ("input");
10477 yyparse ("input");
10478 return 0;
10479}
10480@end verbatim
10481
10482@noindent
10483If the file @file{input} contains
10484
10485@verbatim
10486input:1: Hello,
10487input:2: World!
10488@end verbatim
10489
10490@noindent
0e14ad77 10491then instead of getting the first line twice, you get:
5b066063
AD
10492
10493@example
10494$ @kbd{flex -ofirst-line.c first-line.l}
10495$ @kbd{gcc -ofirst-line first-line.c -ll}
10496$ @kbd{./first-line}
10497input:1: Hello,
10498input:2: World!
10499@end example
10500
0e14ad77
PE
10501Therefore, whenever you change @code{yyin}, you must tell the
10502Lex-generated scanner to discard its current buffer and switch to the
10503new one. This depends upon your implementation of Lex; see its
10504documentation for more. For Flex, it suffices to call
10505@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
10506Flex-generated scanner needs to read from several input streams to
10507handle features like include files, you might consider using Flex
10508functions like @samp{yy_switch_to_buffer} that manipulate multiple
10509input buffers.
5b066063 10510
b165c324
AD
10511If your Flex-generated scanner uses start conditions (@pxref{Start
10512conditions, , Start conditions, flex, The Flex Manual}), you might
10513also want to reset the scanner's state, i.e., go back to the initial
10514start condition, through a call to @samp{BEGIN (0)}.
10515
fef4cb51
AD
10516@node Strings are Destroyed
10517@section Strings are Destroyed
10518
10519@display
c7e441b4 10520My parser seems to destroy old strings, or maybe it loses track of
fef4cb51
AD
10521them. Instead of reporting @samp{"foo", "bar"}, it reports
10522@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
10523@end display
10524
10525This error is probably the single most frequent ``bug report'' sent to
10526Bison lists, but is only concerned with a misunderstanding of the role
8c5b881d 10527of the scanner. Consider the following Lex code:
fef4cb51
AD
10528
10529@verbatim
10530%{
10531#include <stdio.h>
10532char *yylval = NULL;
10533%}
10534%%
10535.* yylval = yytext; return 1;
10536\n /* IGNORE */
10537%%
10538int
10539main ()
10540{
fa7e68c3 10541 /* Similar to using $1, $2 in a Bison action. */
fef4cb51
AD
10542 char *fst = (yylex (), yylval);
10543 char *snd = (yylex (), yylval);
10544 printf ("\"%s\", \"%s\"\n", fst, snd);
10545 return 0;
10546}
10547@end verbatim
10548
10549If you compile and run this code, you get:
10550
10551@example
10552$ @kbd{flex -osplit-lines.c split-lines.l}
10553$ @kbd{gcc -osplit-lines split-lines.c -ll}
10554$ @kbd{printf 'one\ntwo\n' | ./split-lines}
10555"one
10556two", "two"
10557@end example
10558
10559@noindent
10560this is because @code{yytext} is a buffer provided for @emph{reading}
10561in the action, but if you want to keep it, you have to duplicate it
10562(e.g., using @code{strdup}). Note that the output may depend on how
10563your implementation of Lex handles @code{yytext}. For instance, when
10564given the Lex compatibility option @option{-l} (which triggers the
10565option @samp{%array}) Flex generates a different behavior:
10566
10567@example
10568$ @kbd{flex -l -osplit-lines.c split-lines.l}
10569$ @kbd{gcc -osplit-lines split-lines.c -ll}
10570$ @kbd{printf 'one\ntwo\n' | ./split-lines}
10571"two", "two"
10572@end example
10573
10574
2fa09258
AD
10575@node Implementing Gotos/Loops
10576@section Implementing Gotos/Loops
a06ea4aa
AD
10577
10578@display
10579My simple calculator supports variables, assignments, and functions,
2fa09258 10580but how can I implement gotos, or loops?
a06ea4aa
AD
10581@end display
10582
10583Although very pedagogical, the examples included in the document blur
a1c84f45 10584the distinction to make between the parser---whose job is to recover
a06ea4aa 10585the structure of a text and to transmit it to subsequent modules of
a1c84f45 10586the program---and the processing (such as the execution) of this
a06ea4aa
AD
10587structure. This works well with so called straight line programs,
10588i.e., precisely those that have a straightforward execution model:
10589execute simple instructions one after the others.
10590
10591@cindex abstract syntax tree
8a4281b9 10592@cindex AST
a06ea4aa
AD
10593If you want a richer model, you will probably need to use the parser
10594to construct a tree that does represent the structure it has
10595recovered; this tree is usually called the @dfn{abstract syntax tree},
8a4281b9 10596or @dfn{AST} for short. Then, walking through this tree,
a06ea4aa
AD
10597traversing it in various ways, will enable treatments such as its
10598execution or its translation, which will result in an interpreter or a
10599compiler.
10600
10601This topic is way beyond the scope of this manual, and the reader is
10602invited to consult the dedicated literature.
10603
10604
ed2e6384
AD
10605@node Multiple start-symbols
10606@section Multiple start-symbols
10607
10608@display
10609I have several closely related grammars, and I would like to share their
10610implementations. In fact, I could use a single grammar but with
10611multiple entry points.
10612@end display
10613
10614Bison does not support multiple start-symbols, but there is a very
10615simple means to simulate them. If @code{foo} and @code{bar} are the two
10616pseudo start-symbols, then introduce two new tokens, say
10617@code{START_FOO} and @code{START_BAR}, and use them as switches from the
10618real start-symbol:
10619
10620@example
10621%token START_FOO START_BAR;
10622%start start;
10623start: START_FOO foo
10624 | START_BAR bar;
10625@end example
10626
10627These tokens prevents the introduction of new conflicts. As far as the
10628parser goes, that is all that is needed.
10629
10630Now the difficult part is ensuring that the scanner will send these
10631tokens first. If your scanner is hand-written, that should be
10632straightforward. If your scanner is generated by Lex, them there is
10633simple means to do it: recall that anything between @samp{%@{ ... %@}}
10634after the first @code{%%} is copied verbatim in the top of the generated
10635@code{yylex} function. Make sure a variable @code{start_token} is
10636available in the scanner (e.g., a global variable or using
10637@code{%lex-param} etc.), and use the following:
10638
10639@example
10640 /* @r{Prologue.} */
10641%%
10642%@{
10643 if (start_token)
10644 @{
10645 int t = start_token;
10646 start_token = 0;
10647 return t;
10648 @}
10649%@}
10650 /* @r{The rules.} */
10651@end example
10652
10653
55ba27be
AD
10654@node Secure? Conform?
10655@section Secure? Conform?
10656
10657@display
10658Is Bison secure? Does it conform to POSIX?
10659@end display
10660
10661If you're looking for a guarantee or certification, we don't provide it.
10662However, Bison is intended to be a reliable program that conforms to the
8a4281b9 10663POSIX specification for Yacc. If you run into problems,
55ba27be
AD
10664please send us a bug report.
10665
10666@node I can't build Bison
10667@section I can't build Bison
10668
10669@display
8c5b881d
PE
10670I can't build Bison because @command{make} complains that
10671@code{msgfmt} is not found.
55ba27be
AD
10672What should I do?
10673@end display
10674
10675Like most GNU packages with internationalization support, that feature
10676is turned on by default. If you have problems building in the @file{po}
10677subdirectory, it indicates that your system's internationalization
10678support is lacking. You can re-configure Bison with
10679@option{--disable-nls} to turn off this support, or you can install GNU
10680gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
10681Bison. See the file @file{ABOUT-NLS} for more information.
10682
10683
10684@node Where can I find help?
10685@section Where can I find help?
10686
10687@display
10688I'm having trouble using Bison. Where can I find help?
10689@end display
10690
10691First, read this fine manual. Beyond that, you can send mail to
10692@email{help-bison@@gnu.org}. This mailing list is intended to be
10693populated with people who are willing to answer questions about using
10694and installing Bison. Please keep in mind that (most of) the people on
10695the list have aspects of their lives which are not related to Bison (!),
10696so you may not receive an answer to your question right away. This can
10697be frustrating, but please try not to honk them off; remember that any
10698help they provide is purely voluntary and out of the kindness of their
10699hearts.
10700
10701@node Bug Reports
10702@section Bug Reports
10703
10704@display
10705I found a bug. What should I include in the bug report?
10706@end display
10707
10708Before you send a bug report, make sure you are using the latest
10709version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
10710mirrors. Be sure to include the version number in your bug report. If
10711the bug is present in the latest version but not in a previous version,
10712try to determine the most recent version which did not contain the bug.
10713
10714If the bug is parser-related, you should include the smallest grammar
10715you can which demonstrates the bug. The grammar file should also be
10716complete (i.e., I should be able to run it through Bison without having
10717to edit or add anything). The smaller and simpler the grammar, the
10718easier it will be to fix the bug.
10719
10720Include information about your compilation environment, including your
10721operating system's name and version and your compiler's name and
10722version. If you have trouble compiling, you should also include a
10723transcript of the build session, starting with the invocation of
10724`configure'. Depending on the nature of the bug, you may be asked to
10725send additional files as well (such as `config.h' or `config.cache').
10726
10727Patches are most welcome, but not required. That is, do not hesitate to
10728send a bug report just because you can not provide a fix.
10729
10730Send bug reports to @email{bug-bison@@gnu.org}.
10731
8405b70c
PB
10732@node More Languages
10733@section More Languages
55ba27be
AD
10734
10735@display
8405b70c 10736Will Bison ever have C++ and Java support? How about @var{insert your
55ba27be
AD
10737favorite language here}?
10738@end display
10739
8405b70c 10740C++ and Java support is there now, and is documented. We'd love to add other
55ba27be
AD
10741languages; contributions are welcome.
10742
10743@node Beta Testing
10744@section Beta Testing
10745
10746@display
10747What is involved in being a beta tester?
10748@end display
10749
10750It's not terribly involved. Basically, you would download a test
10751release, compile it, and use it to build and run a parser or two. After
10752that, you would submit either a bug report or a message saying that
10753everything is okay. It is important to report successes as well as
10754failures because test releases eventually become mainstream releases,
10755but only if they are adequately tested. If no one tests, development is
10756essentially halted.
10757
10758Beta testers are particularly needed for operating systems to which the
10759developers do not have easy access. They currently have easy access to
10760recent GNU/Linux and Solaris versions. Reports about other operating
10761systems are especially welcome.
10762
10763@node Mailing Lists
10764@section Mailing Lists
10765
10766@display
10767How do I join the help-bison and bug-bison mailing lists?
10768@end display
10769
10770See @url{http://lists.gnu.org/}.
a06ea4aa 10771
d1a1114f
AD
10772@c ================================================= Table of Symbols
10773
342b8b6e 10774@node Table of Symbols
bfa74976
RS
10775@appendix Bison Symbols
10776@cindex Bison symbols, table of
10777@cindex symbols in Bison, table of
10778
18b519c0 10779@deffn {Variable} @@$
3ded9a63 10780In an action, the location of the left-hand side of the rule.
88bce5a2 10781@xref{Locations, , Locations Overview}.
18b519c0 10782@end deffn
3ded9a63 10783
18b519c0 10784@deffn {Variable} @@@var{n}
3ded9a63
AD
10785In an action, the location of the @var{n}-th symbol of the right-hand
10786side of the rule. @xref{Locations, , Locations Overview}.
18b519c0 10787@end deffn
3ded9a63 10788
d013372c
AR
10789@deffn {Variable} @@@var{name}
10790In an action, the location of a symbol addressed by name.
10791@xref{Locations, , Locations Overview}.
10792@end deffn
10793
10794@deffn {Variable} @@[@var{name}]
10795In an action, the location of a symbol addressed by name.
10796@xref{Locations, , Locations Overview}.
10797@end deffn
10798
18b519c0 10799@deffn {Variable} $$
3ded9a63
AD
10800In an action, the semantic value of the left-hand side of the rule.
10801@xref{Actions}.
18b519c0 10802@end deffn
3ded9a63 10803
18b519c0 10804@deffn {Variable} $@var{n}
3ded9a63
AD
10805In an action, the semantic value of the @var{n}-th symbol of the
10806right-hand side of the rule. @xref{Actions}.
18b519c0 10807@end deffn
3ded9a63 10808
d013372c
AR
10809@deffn {Variable} $@var{name}
10810In an action, the semantic value of a symbol addressed by name.
10811@xref{Actions}.
10812@end deffn
10813
10814@deffn {Variable} $[@var{name}]
10815In an action, the semantic value of a symbol addressed by name.
10816@xref{Actions}.
10817@end deffn
10818
dd8d9022
AD
10819@deffn {Delimiter} %%
10820Delimiter used to separate the grammar rule section from the
10821Bison declarations section or the epilogue.
10822@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
18b519c0 10823@end deffn
bfa74976 10824
dd8d9022
AD
10825@c Don't insert spaces, or check the DVI output.
10826@deffn {Delimiter} %@{@var{code}%@}
ff7571c0
JD
10827All code listed between @samp{%@{} and @samp{%@}} is copied verbatim
10828to the parser implementation file. Such code forms the prologue of
10829the grammar file. @xref{Grammar Outline, ,Outline of a Bison
dd8d9022 10830Grammar}.
18b519c0 10831@end deffn
bfa74976 10832
ca2a6d15
PH
10833@deffn {Directive} %?@{@var{expression}@}
10834Predicate actions. This is a type of action clause that may appear in
10835rules. The expression is evaluated, and if false, causes a syntax error. In
8a4281b9 10836GLR parsers during nondeterministic operation,
ca2a6d15
PH
10837this silently causes an alternative parse to die. During deterministic
10838operation, it is the same as the effect of YYERROR.
10839@xref{Semantic Predicates}.
10840
10841This feature is experimental.
10842More user feedback will help to determine whether it should become a permanent
10843feature.
10844@end deffn
10845
dd8d9022
AD
10846@deffn {Construct} /*@dots{}*/
10847Comment delimiters, as in C.
18b519c0 10848@end deffn
bfa74976 10849
dd8d9022
AD
10850@deffn {Delimiter} :
10851Separates a rule's result from its components. @xref{Rules, ,Syntax of
10852Grammar Rules}.
18b519c0 10853@end deffn
bfa74976 10854
dd8d9022
AD
10855@deffn {Delimiter} ;
10856Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 10857@end deffn
bfa74976 10858
dd8d9022
AD
10859@deffn {Delimiter} |
10860Separates alternate rules for the same result nonterminal.
10861@xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 10862@end deffn
bfa74976 10863
12e35840
JD
10864@deffn {Directive} <*>
10865Used to define a default tagged @code{%destructor} or default tagged
10866@code{%printer}.
85894313
JD
10867
10868This feature is experimental.
10869More user feedback will help to determine whether it should become a permanent
10870feature.
10871
12e35840
JD
10872@xref{Destructor Decl, , Freeing Discarded Symbols}.
10873@end deffn
10874
3ebecc24 10875@deffn {Directive} <>
12e35840
JD
10876Used to define a default tagless @code{%destructor} or default tagless
10877@code{%printer}.
85894313
JD
10878
10879This feature is experimental.
10880More user feedback will help to determine whether it should become a permanent
10881feature.
10882
12e35840
JD
10883@xref{Destructor Decl, , Freeing Discarded Symbols}.
10884@end deffn
10885
dd8d9022
AD
10886@deffn {Symbol} $accept
10887The predefined nonterminal whose only rule is @samp{$accept: @var{start}
10888$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
10889Start-Symbol}. It cannot be used in the grammar.
18b519c0 10890@end deffn
bfa74976 10891
136a0f76 10892@deffn {Directive} %code @{@var{code}@}
148d66d8 10893@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
51151d91
JD
10894Insert @var{code} verbatim into the output parser source at the
10895default location or at the location specified by @var{qualifier}.
e0c07222 10896@xref{%code Summary}.
9bc0dd67
JD
10897@end deffn
10898
10899@deffn {Directive} %debug
10900Equip the parser for debugging. @xref{Decl Summary}.
10901@end deffn
10902
91d2c560 10903@ifset defaultprec
22fccf95
PE
10904@deffn {Directive} %default-prec
10905Assign a precedence to rules that lack an explicit @samp{%prec}
10906modifier. @xref{Contextual Precedence, ,Context-Dependent
10907Precedence}.
39a06c25 10908@end deffn
91d2c560 10909@end ifset
39a06c25 10910
148d66d8
JD
10911@deffn {Directive} %define @var{define-variable}
10912@deffnx {Directive} %define @var{define-variable} @var{value}
cf499cff 10913@deffnx {Directive} %define @var{define-variable} "@var{value}"
35c1e5f0 10914Define a variable to adjust Bison's behavior. @xref{%define Summary}.
148d66d8
JD
10915@end deffn
10916
18b519c0 10917@deffn {Directive} %defines
ff7571c0
JD
10918Bison declaration to create a parser header file, which is usually
10919meant for the scanner. @xref{Decl Summary}.
18b519c0 10920@end deffn
6deb4447 10921
02975b9a
JD
10922@deffn {Directive} %defines @var{defines-file}
10923Same as above, but save in the file @var{defines-file}.
10924@xref{Decl Summary}.
10925@end deffn
10926
18b519c0 10927@deffn {Directive} %destructor
258b75ca 10928Specify how the parser should reclaim the memory associated to
fa7e68c3 10929discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 10930@end deffn
72f889cc 10931
18b519c0 10932@deffn {Directive} %dprec
676385e2 10933Bison declaration to assign a precedence to a rule that is used at parse
c827f760 10934time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
8a4281b9 10935GLR Parsers}.
18b519c0 10936@end deffn
676385e2 10937
dd8d9022
AD
10938@deffn {Symbol} $end
10939The predefined token marking the end of the token stream. It cannot be
10940used in the grammar.
10941@end deffn
10942
10943@deffn {Symbol} error
10944A token name reserved for error recovery. This token may be used in
10945grammar rules so as to allow the Bison parser to recognize an error in
10946the grammar without halting the process. In effect, a sentence
10947containing an error may be recognized as valid. On a syntax error, the
742e4900
JD
10948token @code{error} becomes the current lookahead token. Actions
10949corresponding to @code{error} are then executed, and the lookahead
dd8d9022
AD
10950token is reset to the token that originally caused the violation.
10951@xref{Error Recovery}.
18d192f0
AD
10952@end deffn
10953
18b519c0 10954@deffn {Directive} %error-verbose
cf499cff 10955An obsolete directive standing for @samp{%define parse.error verbose}.
18b519c0 10956@end deffn
2a8d363a 10957
02975b9a 10958@deffn {Directive} %file-prefix "@var{prefix}"
72d2299c 10959Bison declaration to set the prefix of the output files. @xref{Decl
d8988b2f 10960Summary}.
18b519c0 10961@end deffn
d8988b2f 10962
18b519c0 10963@deffn {Directive} %glr-parser
8a4281b9
JD
10964Bison declaration to produce a GLR parser. @xref{GLR
10965Parsers, ,Writing GLR Parsers}.
18b519c0 10966@end deffn
676385e2 10967
dd8d9022
AD
10968@deffn {Directive} %initial-action
10969Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
10970@end deffn
10971
e6e704dc
JD
10972@deffn {Directive} %language
10973Specify the programming language for the generated parser.
10974@xref{Decl Summary}.
10975@end deffn
10976
18b519c0 10977@deffn {Directive} %left
d78f0ac9 10978Bison declaration to assign precedence and left associativity to token(s).
bfa74976 10979@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 10980@end deffn
bfa74976 10981
2055a44e
AD
10982@deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
10983Bison declaration to specifying additional arguments that
2a8d363a
AD
10984@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
10985for Pure Parsers}.
18b519c0 10986@end deffn
2a8d363a 10987
18b519c0 10988@deffn {Directive} %merge
676385e2 10989Bison declaration to assign a merging function to a rule. If there is a
fae437e8 10990reduce/reduce conflict with a rule having the same merging function, the
676385e2 10991function is applied to the two semantic values to get a single result.
8a4281b9 10992@xref{GLR Parsers, ,Writing GLR Parsers}.
18b519c0 10993@end deffn
676385e2 10994
02975b9a 10995@deffn {Directive} %name-prefix "@var{prefix}"
72d2299c 10996Bison declaration to rename the external symbols. @xref{Decl Summary}.
18b519c0 10997@end deffn
d8988b2f 10998
91d2c560 10999@ifset defaultprec
22fccf95
PE
11000@deffn {Directive} %no-default-prec
11001Do not assign a precedence to rules that lack an explicit @samp{%prec}
11002modifier. @xref{Contextual Precedence, ,Context-Dependent
11003Precedence}.
11004@end deffn
91d2c560 11005@end ifset
22fccf95 11006
18b519c0 11007@deffn {Directive} %no-lines
931c7513 11008Bison declaration to avoid generating @code{#line} directives in the
ff7571c0 11009parser implementation file. @xref{Decl Summary}.
18b519c0 11010@end deffn
931c7513 11011
18b519c0 11012@deffn {Directive} %nonassoc
d78f0ac9 11013Bison declaration to assign precedence and nonassociativity to token(s).
bfa74976 11014@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 11015@end deffn
bfa74976 11016
02975b9a 11017@deffn {Directive} %output "@var{file}"
ff7571c0
JD
11018Bison declaration to set the name of the parser implementation file.
11019@xref{Decl Summary}.
18b519c0 11020@end deffn
d8988b2f 11021
2055a44e
AD
11022@deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
11023Bison declaration to specify additional arguments that both
11024@code{yylex} and @code{yyparse} should accept. @xref{Parser Function,, The
11025Parser Function @code{yyparse}}.
11026@end deffn
11027
11028@deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
11029Bison declaration to specify additional arguments that @code{yyparse}
11030should accept. @xref{Parser Function,, The Parser Function @code{yyparse}}.
18b519c0 11031@end deffn
2a8d363a 11032
18b519c0 11033@deffn {Directive} %prec
bfa74976
RS
11034Bison declaration to assign a precedence to a specific rule.
11035@xref{Contextual Precedence, ,Context-Dependent Precedence}.
18b519c0 11036@end deffn
bfa74976 11037
d78f0ac9
AD
11038@deffn {Directive} %precedence
11039Bison declaration to assign precedence to token(s), but no associativity
11040@xref{Precedence Decl, ,Operator Precedence}.
11041@end deffn
11042
18b519c0 11043@deffn {Directive} %pure-parser
35c1e5f0
JD
11044Deprecated version of @samp{%define api.pure} (@pxref{%define
11045Summary,,api.pure}), for which Bison is more careful to warn about
11046unreasonable usage.
18b519c0 11047@end deffn
bfa74976 11048
b50d2359 11049@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
11050Require version @var{version} or higher of Bison. @xref{Require Decl, ,
11051Require a Version of Bison}.
b50d2359
AD
11052@end deffn
11053
18b519c0 11054@deffn {Directive} %right
d78f0ac9 11055Bison declaration to assign precedence and right associativity to token(s).
bfa74976 11056@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 11057@end deffn
bfa74976 11058
e6e704dc
JD
11059@deffn {Directive} %skeleton
11060Specify the skeleton to use; usually for development.
11061@xref{Decl Summary}.
11062@end deffn
11063
18b519c0 11064@deffn {Directive} %start
704a47c4
AD
11065Bison declaration to specify the start symbol. @xref{Start Decl, ,The
11066Start-Symbol}.
18b519c0 11067@end deffn
bfa74976 11068
18b519c0 11069@deffn {Directive} %token
bfa74976
RS
11070Bison declaration to declare token(s) without specifying precedence.
11071@xref{Token Decl, ,Token Type Names}.
18b519c0 11072@end deffn
bfa74976 11073
18b519c0 11074@deffn {Directive} %token-table
ff7571c0
JD
11075Bison declaration to include a token name table in the parser
11076implementation file. @xref{Decl Summary}.
18b519c0 11077@end deffn
931c7513 11078
18b519c0 11079@deffn {Directive} %type
704a47c4
AD
11080Bison declaration to declare nonterminals. @xref{Type Decl,
11081,Nonterminal Symbols}.
18b519c0 11082@end deffn
bfa74976 11083
dd8d9022
AD
11084@deffn {Symbol} $undefined
11085The predefined token onto which all undefined values returned by
11086@code{yylex} are mapped. It cannot be used in the grammar, rather, use
11087@code{error}.
11088@end deffn
11089
18b519c0 11090@deffn {Directive} %union
bfa74976
RS
11091Bison declaration to specify several possible data types for semantic
11092values. @xref{Union Decl, ,The Collection of Value Types}.
18b519c0 11093@end deffn
bfa74976 11094
dd8d9022
AD
11095@deffn {Macro} YYABORT
11096Macro to pretend that an unrecoverable syntax error has occurred, by
11097making @code{yyparse} return 1 immediately. The error reporting
11098function @code{yyerror} is not called. @xref{Parser Function, ,The
11099Parser Function @code{yyparse}}.
8405b70c
PB
11100
11101For Java parsers, this functionality is invoked using @code{return YYABORT;}
11102instead.
dd8d9022 11103@end deffn
3ded9a63 11104
dd8d9022
AD
11105@deffn {Macro} YYACCEPT
11106Macro to pretend that a complete utterance of the language has been
11107read, by making @code{yyparse} return 0 immediately.
11108@xref{Parser Function, ,The Parser Function @code{yyparse}}.
8405b70c
PB
11109
11110For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
11111instead.
dd8d9022 11112@end deffn
bfa74976 11113
dd8d9022 11114@deffn {Macro} YYBACKUP
742e4900 11115Macro to discard a value from the parser stack and fake a lookahead
dd8d9022 11116token. @xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 11117@end deffn
bfa74976 11118
dd8d9022 11119@deffn {Variable} yychar
32c29292 11120External integer variable that contains the integer value of the
742e4900 11121lookahead token. (In a pure parser, it is a local variable within
dd8d9022
AD
11122@code{yyparse}.) Error-recovery rule actions may examine this variable.
11123@xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 11124@end deffn
bfa74976 11125
dd8d9022
AD
11126@deffn {Variable} yyclearin
11127Macro used in error-recovery rule actions. It clears the previous
742e4900 11128lookahead token. @xref{Error Recovery}.
18b519c0 11129@end deffn
bfa74976 11130
dd8d9022
AD
11131@deffn {Macro} YYDEBUG
11132Macro to define to equip the parser with tracing code. @xref{Tracing,
11133,Tracing Your Parser}.
18b519c0 11134@end deffn
bfa74976 11135
dd8d9022
AD
11136@deffn {Variable} yydebug
11137External integer variable set to zero by default. If @code{yydebug}
11138is given a nonzero value, the parser will output information on input
11139symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
18b519c0 11140@end deffn
bfa74976 11141
dd8d9022
AD
11142@deffn {Macro} yyerrok
11143Macro to cause parser to recover immediately to its normal mode
11144after a syntax error. @xref{Error Recovery}.
11145@end deffn
11146
11147@deffn {Macro} YYERROR
11148Macro to pretend that a syntax error has just been detected: call
11149@code{yyerror} and then perform normal error recovery if possible
11150(@pxref{Error Recovery}), or (if recovery is impossible) make
11151@code{yyparse} return 1. @xref{Error Recovery}.
8405b70c
PB
11152
11153For Java parsers, this functionality is invoked using @code{return YYERROR;}
11154instead.
dd8d9022
AD
11155@end deffn
11156
11157@deffn {Function} yyerror
11158User-supplied function to be called by @code{yyparse} on error.
71b00ed8 11159@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
dd8d9022
AD
11160@end deffn
11161
11162@deffn {Macro} YYERROR_VERBOSE
71b00ed8
AD
11163An obsolete macro used in the @file{yacc.c} skeleton, that you define
11164with @code{#define} in the prologue to request verbose, specific error
11165message strings when @code{yyerror} is called. It doesn't matter what
11166definition you use for @code{YYERROR_VERBOSE}, just whether you define
cf499cff 11167it. Using @samp{%define parse.error verbose} is preferred
31b850d2 11168(@pxref{Error Reporting, ,The Error Reporting Function @code{yyerror}}).
dd8d9022
AD
11169@end deffn
11170
11171@deffn {Macro} YYINITDEPTH
11172Macro for specifying the initial size of the parser stack.
1a059451 11173@xref{Memory Management}.
dd8d9022
AD
11174@end deffn
11175
11176@deffn {Function} yylex
11177User-supplied lexical analyzer function, called with no arguments to get
11178the next token. @xref{Lexical, ,The Lexical Analyzer Function
11179@code{yylex}}.
11180@end deffn
11181
11182@deffn {Macro} YYLEX_PARAM
11183An obsolete macro for specifying an extra argument (or list of extra
32c29292 11184arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
dd8d9022
AD
11185macro is deprecated, and is supported only for Yacc like parsers.
11186@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
11187@end deffn
11188
11189@deffn {Variable} yylloc
11190External variable in which @code{yylex} should place the line and column
11191numbers associated with a token. (In a pure parser, it is a local
11192variable within @code{yyparse}, and its address is passed to
32c29292
JD
11193@code{yylex}.)
11194You can ignore this variable if you don't use the @samp{@@} feature in the
11195grammar actions.
11196@xref{Token Locations, ,Textual Locations of Tokens}.
742e4900 11197In semantic actions, it stores the location of the lookahead token.
32c29292 11198@xref{Actions and Locations, ,Actions and Locations}.
dd8d9022
AD
11199@end deffn
11200
11201@deffn {Type} YYLTYPE
11202Data type of @code{yylloc}; by default, a structure with four
11203members. @xref{Location Type, , Data Types of Locations}.
11204@end deffn
11205
11206@deffn {Variable} yylval
11207External variable in which @code{yylex} should place the semantic
11208value associated with a token. (In a pure parser, it is a local
11209variable within @code{yyparse}, and its address is passed to
32c29292
JD
11210@code{yylex}.)
11211@xref{Token Values, ,Semantic Values of Tokens}.
742e4900 11212In semantic actions, it stores the semantic value of the lookahead token.
32c29292 11213@xref{Actions, ,Actions}.
dd8d9022
AD
11214@end deffn
11215
11216@deffn {Macro} YYMAXDEPTH
1a059451
PE
11217Macro for specifying the maximum size of the parser stack. @xref{Memory
11218Management}.
dd8d9022
AD
11219@end deffn
11220
11221@deffn {Variable} yynerrs
8a2800e7 11222Global variable which Bison increments each time it reports a syntax error.
f4101aa6 11223(In a pure parser, it is a local variable within @code{yyparse}. In a
9987d1b3 11224pure push parser, it is a member of yypstate.)
dd8d9022
AD
11225@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
11226@end deffn
11227
11228@deffn {Function} yyparse
11229The parser function produced by Bison; call this function to start
11230parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
11231@end deffn
11232
9987d1b3 11233@deffn {Function} yypstate_delete
f4101aa6 11234The function to delete a parser instance, produced by Bison in push mode;
9987d1b3 11235call this function to delete the memory associated with a parser.
f4101aa6 11236@xref{Parser Delete Function, ,The Parser Delete Function
9987d1b3 11237@code{yypstate_delete}}.
59da312b
JD
11238(The current push parsing interface is experimental and may evolve.
11239More user feedback will help to stabilize it.)
9987d1b3
JD
11240@end deffn
11241
11242@deffn {Function} yypstate_new
f4101aa6 11243The function to create a parser instance, produced by Bison in push mode;
9987d1b3 11244call this function to create a new parser.
f4101aa6 11245@xref{Parser Create Function, ,The Parser Create Function
9987d1b3 11246@code{yypstate_new}}.
59da312b
JD
11247(The current push parsing interface is experimental and may evolve.
11248More user feedback will help to stabilize it.)
9987d1b3
JD
11249@end deffn
11250
11251@deffn {Function} yypull_parse
f4101aa6
AD
11252The parser function produced by Bison in push mode; call this function to
11253parse the rest of the input stream.
11254@xref{Pull Parser Function, ,The Pull Parser Function
9987d1b3 11255@code{yypull_parse}}.
59da312b
JD
11256(The current push parsing interface is experimental and may evolve.
11257More user feedback will help to stabilize it.)
9987d1b3
JD
11258@end deffn
11259
11260@deffn {Function} yypush_parse
f4101aa6
AD
11261The parser function produced by Bison in push mode; call this function to
11262parse a single token. @xref{Push Parser Function, ,The Push Parser Function
9987d1b3 11263@code{yypush_parse}}.
59da312b
JD
11264(The current push parsing interface is experimental and may evolve.
11265More user feedback will help to stabilize it.)
9987d1b3
JD
11266@end deffn
11267
dd8d9022
AD
11268@deffn {Macro} YYPARSE_PARAM
11269An obsolete macro for specifying the name of a parameter that
11270@code{yyparse} should accept. The use of this macro is deprecated, and
11271is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
11272Conventions for Pure Parsers}.
11273@end deffn
11274
11275@deffn {Macro} YYRECOVERING
02103984
PE
11276The expression @code{YYRECOVERING ()} yields 1 when the parser
11277is recovering from a syntax error, and 0 otherwise.
11278@xref{Action Features, ,Special Features for Use in Actions}.
dd8d9022
AD
11279@end deffn
11280
11281@deffn {Macro} YYSTACK_USE_ALLOCA
eb45ef3b
JD
11282Macro used to control the use of @code{alloca} when the
11283deterministic parser in C needs to extend its stacks. If defined to 0,
d7e14fc0
PE
11284the parser will use @code{malloc} to extend its stacks. If defined to
112851, the parser will use @code{alloca}. Values other than 0 and 1 are
11286reserved for future Bison extensions. If not defined,
11287@code{YYSTACK_USE_ALLOCA} defaults to 0.
11288
55289366 11289In the all-too-common case where your code may run on a host with a
d7e14fc0
PE
11290limited stack and with unreliable stack-overflow checking, you should
11291set @code{YYMAXDEPTH} to a value that cannot possibly result in
11292unchecked stack overflow on any of your target hosts when
11293@code{alloca} is called. You can inspect the code that Bison
11294generates in order to determine the proper numeric values. This will
11295require some expertise in low-level implementation details.
dd8d9022
AD
11296@end deffn
11297
11298@deffn {Type} YYSTYPE
11299Data type of semantic values; @code{int} by default.
11300@xref{Value Type, ,Data Types of Semantic Values}.
18b519c0 11301@end deffn
bfa74976 11302
342b8b6e 11303@node Glossary
bfa74976
RS
11304@appendix Glossary
11305@cindex glossary
11306
11307@table @asis
eb45ef3b
JD
11308@item Accepting State
11309A state whose only action is the accept action.
11310The accepting state is thus a consistent state.
11311@xref{Understanding,,}.
11312
8a4281b9 11313@item Backus-Naur Form (BNF; also called ``Backus Normal Form'')
c827f760
PE
11314Formal method of specifying context-free grammars originally proposed
11315by John Backus, and slightly improved by Peter Naur in his 1960-01-02
11316committee document contributing to what became the Algol 60 report.
11317@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976 11318
eb45ef3b 11319@item Consistent State
35c1e5f0
JD
11320A state containing only one possible action. @xref{%define
11321Summary,,lr.default-reductions}.
eb45ef3b 11322
bfa74976
RS
11323@item Context-free grammars
11324Grammars specified as rules that can be applied regardless of context.
11325Thus, if there is a rule which says that an integer can be used as an
11326expression, integers are allowed @emph{anywhere} an expression is
89cab50d
AD
11327permitted. @xref{Language and Grammar, ,Languages and Context-Free
11328Grammars}.
bfa74976 11329
110ef36a
JD
11330@item Default Reduction
11331The reduction that a parser should perform if the current parser state
35c1e5f0
JD
11332contains no other action for the lookahead token. In permitted parser
11333states, Bison declares the reduction with the largest lookahead set to
11334be the default reduction and removes that lookahead set.
11335@xref{%define Summary,,lr.default-reductions}.
eb45ef3b 11336
bfa74976
RS
11337@item Dynamic allocation
11338Allocation of memory that occurs during execution, rather than at
11339compile time or on entry to a function.
11340
11341@item Empty string
11342Analogous to the empty set in set theory, the empty string is a
11343character string of length zero.
11344
11345@item Finite-state stack machine
11346A ``machine'' that has discrete states in which it is said to exist at
11347each instant in time. As input to the machine is processed, the
11348machine moves from state to state as specified by the logic of the
11349machine. In the case of the parser, the input is the language being
11350parsed, and the states correspond to various stages in the grammar
c827f760 11351rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976 11352
8a4281b9 11353@item Generalized LR (GLR)
676385e2 11354A parsing algorithm that can handle all context-free grammars, including those
8a4281b9 11355that are not LR(1). It resolves situations that Bison's
eb45ef3b 11356deterministic parsing
676385e2
PH
11357algorithm cannot by effectively splitting off multiple parsers, trying all
11358possible parsers, and discarding those that fail in the light of additional
c827f760 11359right context. @xref{Generalized LR Parsing, ,Generalized
8a4281b9 11360LR Parsing}.
676385e2 11361
bfa74976
RS
11362@item Grouping
11363A language construct that is (in general) grammatically divisible;
c827f760 11364for example, `expression' or `declaration' in C@.
bfa74976
RS
11365@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11366
8a4281b9 11367@item IELR(1)
35c1e5f0
JD
11368A minimal LR(1) parser table generation algorithm. That is, given any
11369context-free grammar, IELR(1) generates parser tables with the full
11370language recognition power of canonical LR(1) but with nearly the same
11371number of parser states as LALR(1). This reduction in parser states
11372is often an order of magnitude. More importantly, because canonical
11373LR(1)'s extra parser states may contain duplicate conflicts in the
11374case of non-LR(1) grammars, the number of conflicts for IELR(1) is
11375often an order of magnitude less as well. This can significantly
11376reduce the complexity of developing of a grammar. @xref{%define
11377Summary,,lr.type}.
eb45ef3b 11378
bfa74976
RS
11379@item Infix operator
11380An arithmetic operator that is placed between the operands on which it
11381performs some operation.
11382
11383@item Input stream
11384A continuous flow of data between devices or programs.
11385
8a4281b9 11386@item LAC (Lookahead Correction)
fcf834f9 11387A parsing mechanism that fixes the problem of delayed syntax error
35c1e5f0
JD
11388detection, which is caused by LR state merging, default reductions,
11389and the use of @code{%nonassoc}. Delayed syntax error detection
11390results in unexpected semantic actions, initiation of error recovery
11391in the wrong syntactic context, and an incorrect list of expected
11392tokens in a verbose syntax error message. @xref{%define
11393Summary,,parse.lac}.
fcf834f9 11394
bfa74976
RS
11395@item Language construct
11396One of the typical usage schemas of the language. For example, one of
11397the constructs of the C language is the @code{if} statement.
11398@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11399
11400@item Left associativity
11401Operators having left associativity are analyzed from left to right:
11402@samp{a+b+c} first computes @samp{a+b} and then combines with
11403@samp{c}. @xref{Precedence, ,Operator Precedence}.
11404
11405@item Left recursion
89cab50d
AD
11406A rule whose result symbol is also its first component symbol; for
11407example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
11408Rules}.
bfa74976
RS
11409
11410@item Left-to-right parsing
11411Parsing a sentence of a language by analyzing it token by token from
c827f760 11412left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
11413
11414@item Lexical analyzer (scanner)
11415A function that reads an input stream and returns tokens one by one.
11416@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
11417
11418@item Lexical tie-in
11419A flag, set by actions in the grammar rules, which alters the way
11420tokens are parsed. @xref{Lexical Tie-ins}.
11421
931c7513 11422@item Literal string token
14ded682 11423A token which consists of two or more fixed characters. @xref{Symbols}.
931c7513 11424
742e4900
JD
11425@item Lookahead token
11426A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
89cab50d 11427Tokens}.
bfa74976 11428
8a4281b9 11429@item LALR(1)
bfa74976 11430The class of context-free grammars that Bison (like most other parser
8a4281b9 11431generators) can handle by default; a subset of LR(1).
eb45ef3b 11432@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
bfa74976 11433
8a4281b9 11434@item LR(1)
bfa74976 11435The class of context-free grammars in which at most one token of
742e4900 11436lookahead is needed to disambiguate the parsing of any piece of input.
bfa74976
RS
11437
11438@item Nonterminal symbol
11439A grammar symbol standing for a grammatical construct that can
11440be expressed through rules in terms of smaller constructs; in other
11441words, a construct that is not a token. @xref{Symbols}.
11442
bfa74976
RS
11443@item Parser
11444A function that recognizes valid sentences of a language by analyzing
11445the syntax structure of a set of tokens passed to it from a lexical
11446analyzer.
11447
11448@item Postfix operator
11449An arithmetic operator that is placed after the operands upon which it
11450performs some operation.
11451
11452@item Reduction
11453Replacing a string of nonterminals and/or terminals with a single
89cab50d 11454nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
c827f760 11455Parser Algorithm}.
bfa74976
RS
11456
11457@item Reentrant
11458A reentrant subprogram is a subprogram which can be in invoked any
11459number of times in parallel, without interference between the various
11460invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
11461
11462@item Reverse polish notation
11463A language in which all operators are postfix operators.
11464
11465@item Right recursion
89cab50d
AD
11466A rule whose result symbol is also its last component symbol; for
11467example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
11468Rules}.
bfa74976
RS
11469
11470@item Semantics
11471In computer languages, the semantics are specified by the actions
11472taken for each instance of the language, i.e., the meaning of
11473each statement. @xref{Semantics, ,Defining Language Semantics}.
11474
11475@item Shift
11476A parser is said to shift when it makes the choice of analyzing
11477further input from the stream rather than reducing immediately some
c827f760 11478already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
11479
11480@item Single-character literal
11481A single character that is recognized and interpreted as is.
11482@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
11483
11484@item Start symbol
11485The nonterminal symbol that stands for a complete valid utterance in
11486the language being parsed. The start symbol is usually listed as the
13863333 11487first nonterminal symbol in a language specification.
bfa74976
RS
11488@xref{Start Decl, ,The Start-Symbol}.
11489
11490@item Symbol table
11491A data structure where symbol names and associated data are stored
11492during parsing to allow for recognition and use of existing
11493information in repeated uses of a symbol. @xref{Multi-function Calc}.
11494
6e649e65
PE
11495@item Syntax error
11496An error encountered during parsing of an input stream due to invalid
11497syntax. @xref{Error Recovery}.
11498
bfa74976
RS
11499@item Token
11500A basic, grammatically indivisible unit of a language. The symbol
11501that describes a token in the grammar is a terminal symbol.
11502The input of the Bison parser is a stream of tokens which comes from
11503the lexical analyzer. @xref{Symbols}.
11504
11505@item Terminal symbol
89cab50d
AD
11506A grammar symbol that has no rules in the grammar and therefore is
11507grammatically indivisible. The piece of text it represents is a token.
11508@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
11509@end table
11510
342b8b6e 11511@node Copying This Manual
f2b5126e 11512@appendix Copying This Manual
f2b5126e
PB
11513@include fdl.texi
11514
5e528941
JD
11515@node Bibliography
11516@unnumbered Bibliography
11517
11518@table @asis
11519@item [Denny 2008]
11520Joel E. Denny and Brian A. Malloy, IELR(1): Practical LR(1) Parser Tables
11521for Non-LR(1) Grammars with Conflict Resolution, in @cite{Proceedings of the
115222008 ACM Symposium on Applied Computing} (SAC'08), ACM, New York, NY, USA,
11523pp.@: 240--245. @uref{http://dx.doi.org/10.1145/1363686.1363747}
11524
11525@item [Denny 2010 May]
11526Joel E. Denny, PSLR(1): Pseudo-Scannerless Minimal LR(1) for the
11527Deterministic Parsing of Composite Languages, Ph.D. Dissertation, Clemson
11528University, Clemson, SC, USA (May 2010).
11529@uref{http://proquest.umi.com/pqdlink?did=2041473591&Fmt=7&clientId=79356&RQT=309&VName=PQD}
11530
11531@item [Denny 2010 November]
11532Joel E. Denny and Brian A. Malloy, The IELR(1) Algorithm for Generating
11533Minimal LR(1) Parser Tables for Non-LR(1) Grammars with Conflict Resolution,
11534in @cite{Science of Computer Programming}, Vol.@: 75, Issue 11 (November
115352010), pp.@: 943--979. @uref{http://dx.doi.org/10.1016/j.scico.2009.08.001}
11536
11537@item [DeRemer 1982]
11538Frank DeRemer and Thomas Pennello, Efficient Computation of LALR(1)
11539Look-Ahead Sets, in @cite{ACM Transactions on Programming Languages and
11540Systems}, Vol.@: 4, No.@: 4 (October 1982), pp.@:
11541615--649. @uref{http://dx.doi.org/10.1145/69622.357187}
11542
11543@item [Knuth 1965]
11544Donald E. Knuth, On the Translation of Languages from Left to Right, in
11545@cite{Information and Control}, Vol.@: 8, Issue 6 (December 1965), pp.@:
11546607--639. @uref{http://dx.doi.org/10.1016/S0019-9958(65)90426-2}
11547
11548@item [Scott 2000]
11549Elizabeth Scott, Adrian Johnstone, and Shamsa Sadaf Hussain,
11550@cite{Tomita-Style Generalised LR Parsers}, Royal Holloway, University of
11551London, Department of Computer Science, TR-00-12 (December 2000).
11552@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps}
11553@end table
11554
342b8b6e 11555@node Index
bfa74976
RS
11556@unnumbered Index
11557
11558@printindex cp
11559
bfa74976 11560@bye
a06ea4aa 11561
6b5a0de9
AD
11562@c LocalWords: texinfo setfilename settitle setchapternewpage finalout texi FSF
11563@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex FSF's
11564@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry Naur
11565@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa Multi
11566@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc multi
11567@c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex defaultprec Donnelly Gotos
11568@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref yypush
11569@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex lr
11570@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge POSIX
11571@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG yypull
11572@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit nonfree
11573@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok rr
11574@c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln Stallman Destructor
11575@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym enum
11576@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof Lex
11577@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum DOTDOT
11578@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype Unary
11579@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs nonterminal
11580@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES reentrant
11581@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param yypstate
11582@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP subrange
11583@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword loc
11584@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH inline
11585@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype Lookahead
11586@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args Autoconf
11587@c LocalWords: infile ypp yxx outfile itemx tex leaderfill Troubleshouting sqrt
11588@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll lookahead
11589@c LocalWords: nbar yytext fst snd osplit ntwo strdup AST Troublereporting th
11590@c LocalWords: YYSTACK DVI fdl printindex IELR nondeterministic nonterminals ps
fcf834f9 11591@c LocalWords: subexpressions declarator nondeferred config libintl postfix LAC
6b5a0de9 11592@c LocalWords: preprocessor nonpositive unary nonnumeric typedef extern rhs
ff7571c0 11593@c LocalWords: yytokentype destructor multicharacter nonnull EBCDIC
6b5a0de9
AD
11594@c LocalWords: lvalue nonnegative XNUM CHR chr TAGLESS tagless stdout api TOK
11595@c LocalWords: destructors Reentrancy nonreentrant subgrammar nonassociative
11596@c LocalWords: deffnx namespace xml goto lalr ielr runtime lex yacc yyps env
11597@c LocalWords: yystate variadic Unshift NLS gettext po UTF Automake LOCALEDIR
11598@c LocalWords: YYENABLE bindtextdomain Makefile DEFS CPPFLAGS DBISON DeRemer
11599@c LocalWords: autoreconf Pennello multisets nondeterminism Generalised baz
11600@c LocalWords: redeclare automata Dparse localedir datadir XSLT midrule Wno
ff7571c0 11601@c LocalWords: Graphviz multitable headitem hh basename Doxygen fno
6b5a0de9
AD
11602@c LocalWords: doxygen ival sval deftypemethod deallocate pos deftypemethodx
11603@c LocalWords: Ctor defcv defcvx arg accessors arithmetics CPP ifndef CALCXX
11604@c LocalWords: lexer's calcxx bool LPAREN RPAREN deallocation cerrno climits
11605@c LocalWords: cstdlib Debian undef yywrap unput noyywrap nounput zA yyleng
11606@c LocalWords: errno strtol ERANGE str strerror iostream argc argv Javadoc
11607@c LocalWords: bytecode initializers superclass stype ASTNode autoboxing nls
11608@c LocalWords: toString deftypeivar deftypeivarx deftypeop YYParser strictfp
11609@c LocalWords: superclasses boolean getErrorVerbose setErrorVerbose deftypecv
11610@c LocalWords: getDebugStream setDebugStream getDebugLevel setDebugLevel url
11611@c LocalWords: bisonVersion deftypecvx bisonSkeleton getStartPos getEndPos
679e9935 11612@c LocalWords: getLVal defvar deftypefn deftypefnx gotos msgfmt Corbett
6b5a0de9 11613@c LocalWords: subdirectory Solaris nonassociativity
e944aaff
AD
11614
11615@c Local Variables:
11616@c ispell-dictionary: "american"
11617@c fill-column: 76
11618@c End: