]> git.saurik.com Git - bison.git/blame_incremental - doc/bison.texinfo
Document named references.
[bison.git] / doc / bison.texinfo
... / ...
CommitLineData
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
4@include version.texi
5@settitle Bison @value{VERSION}
6@setchapternewpage odd
7
8@finalout
9
10@c SMALL BOOK version
11@c This edition has been formatted so that you can format and print it in
12@c the smallbook format.
13@c @smallbook
14
15@c Set following if you want to document %default-prec and %no-default-prec.
16@c This feature is experimental and may change in future Bison versions.
17@c @set defaultprec
18
19@ifnotinfo
20@syncodeindex fn cp
21@syncodeindex vr cp
22@syncodeindex tp cp
23@end ifnotinfo
24@ifinfo
25@synindex fn cp
26@synindex vr cp
27@synindex tp cp
28@end ifinfo
29@comment %**end of header
30
31@copying
32
33This manual (@value{UPDATED}) is for @acronym{GNU} Bison (version
34@value{VERSION}), the @acronym{GNU} parser generator.
35
36Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
371999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Free
38Software Foundation, Inc.
39
40@quotation
41Permission is granted to copy, distribute and/or modify this document
42under the terms of the @acronym{GNU} Free Documentation License,
43Version 1.2 or any later version published by the Free Software
44Foundation; with no Invariant Sections, with the Front-Cover texts
45being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
46(a) below. A copy of the license is included in the section entitled
47``@acronym{GNU} Free Documentation License.''
48
49(a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
50modify this @acronym{GNU} manual. Buying copies from the @acronym{FSF}
51supports it in developing @acronym{GNU} and promoting software
52freedom.''
53@end quotation
54@end copying
55
56@dircategory Software development
57@direntry
58* bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
59@end direntry
60
61@titlepage
62@title Bison
63@subtitle The Yacc-compatible Parser Generator
64@subtitle @value{UPDATED}, Bison Version @value{VERSION}
65
66@author by Charles Donnelly and Richard Stallman
67
68@page
69@vskip 0pt plus 1filll
70@insertcopying
71@sp 2
72Published by the Free Software Foundation @*
7351 Franklin Street, Fifth Floor @*
74Boston, MA 02110-1301 USA @*
75Printed copies are available from the Free Software Foundation.@*
76@acronym{ISBN} 1-882114-44-2
77@sp 2
78Cover art by Etienne Suvasa.
79@end titlepage
80
81@contents
82
83@ifnottex
84@node Top
85@top Bison
86@insertcopying
87@end ifnottex
88
89@menu
90* Introduction::
91* Conditions::
92* Copying:: The @acronym{GNU} General Public License says
93 how you can copy and share Bison.
94
95Tutorial sections:
96* Concepts:: Basic concepts for understanding Bison.
97* Examples:: Three simple explained examples of using Bison.
98
99Reference sections:
100* Grammar File:: Writing Bison declarations and rules.
101* Interface:: C-language interface to the parser function @code{yyparse}.
102* Algorithm:: How the Bison parser works at run-time.
103* Error Recovery:: Writing rules for error recovery.
104* Context Dependency:: What to do if your language syntax is too
105 messy for Bison to handle straightforwardly.
106* Debugging:: Understanding or debugging Bison parsers.
107* Invocation:: How to run Bison (to produce the parser source file).
108* Other Languages:: Creating C++ and Java parsers.
109* FAQ:: Frequently Asked Questions
110* Table of Symbols:: All the keywords of the Bison language are explained.
111* Glossary:: Basic concepts are explained.
112* Copying This Manual:: License for copying this manual.
113* Index:: Cross-references to the text.
114
115@detailmenu
116 --- The Detailed Node Listing ---
117
118The Concepts of Bison
119
120* Language and Grammar:: Languages and context-free grammars,
121 as mathematical ideas.
122* Grammar in Bison:: How we represent grammars for Bison's sake.
123* Semantic Values:: Each token or syntactic grouping can have
124 a semantic value (the value of an integer,
125 the name of an identifier, etc.).
126* Semantic Actions:: Each rule can have an action containing C code.
127* GLR Parsers:: Writing parsers for general context-free languages.
128* Locations Overview:: Tracking Locations.
129* Bison Parser:: What are Bison's input and output,
130 how is the output used?
131* Stages:: Stages in writing and running Bison grammars.
132* Grammar Layout:: Overall structure of a Bison grammar file.
133
134Writing @acronym{GLR} Parsers
135
136* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
137* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
138* GLR Semantic Actions:: Deferred semantic actions have special concerns.
139* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
140
141Examples
142
143* RPN Calc:: Reverse polish notation calculator;
144 a first example with no operator precedence.
145* Infix Calc:: Infix (algebraic) notation calculator.
146 Operator precedence is introduced.
147* Simple Error Recovery:: Continuing after syntax errors.
148* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
149* Multi-function Calc:: Calculator with memory and trig functions.
150 It uses multiple data-types for semantic values.
151* Exercises:: Ideas for improving the multi-function calculator.
152
153Reverse Polish Notation Calculator
154
155* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
156* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
157* Rpcalc Lexer:: The lexical analyzer.
158* Rpcalc Main:: The controlling function.
159* Rpcalc Error:: The error reporting function.
160* Rpcalc Generate:: Running Bison on the grammar file.
161* Rpcalc Compile:: Run the C compiler on the output code.
162
163Grammar Rules for @code{rpcalc}
164
165* Rpcalc Input::
166* Rpcalc Line::
167* Rpcalc Expr::
168
169Location Tracking Calculator: @code{ltcalc}
170
171* Ltcalc Declarations:: Bison and C declarations for ltcalc.
172* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
173* Ltcalc Lexer:: The lexical analyzer.
174
175Multi-Function Calculator: @code{mfcalc}
176
177* Mfcalc Declarations:: Bison declarations for multi-function calculator.
178* Mfcalc Rules:: Grammar rules for the calculator.
179* Mfcalc Symbol Table:: Symbol table management subroutines.
180
181Bison Grammar Files
182
183* Grammar Outline:: Overall layout of the grammar file.
184* Symbols:: Terminal and nonterminal symbols.
185* Rules:: How to write grammar rules.
186* Recursion:: Writing recursive rules.
187* Semantics:: Semantic values and actions.
188* Locations:: Locations and actions.
189* Declarations:: All kinds of Bison declarations are described here.
190* Multiple Parsers:: Putting more than one Bison parser in one program.
191
192Outline of a Bison Grammar
193
194* Prologue:: Syntax and usage of the prologue.
195* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
196* Bison Declarations:: Syntax and usage of the Bison declarations section.
197* Grammar Rules:: Syntax and usage of the grammar rules section.
198* Epilogue:: Syntax and usage of the epilogue.
199
200Defining Language Semantics
201
202* Value Type:: Specifying one data type for all semantic values.
203* Multiple Types:: Specifying several alternative data types.
204* Actions:: An action is the semantic definition of a grammar rule.
205* Action Types:: Specifying data types for actions to operate on.
206* Mid-Rule Actions:: Most actions go at the end of a rule.
207 This says when, why and how to use the exceptional
208 action in the middle of a rule.
209* Named References:: Using named references in actions.
210
211Tracking Locations
212
213* Location Type:: Specifying a data type for locations.
214* Actions and Locations:: Using locations in actions.
215* Location Default Action:: Defining a general way to compute locations.
216
217Bison Declarations
218
219* Require Decl:: Requiring a Bison version.
220* Token Decl:: Declaring terminal symbols.
221* Precedence Decl:: Declaring terminals with precedence and associativity.
222* Union Decl:: Declaring the set of all semantic value types.
223* Type Decl:: Declaring the choice of type for a nonterminal symbol.
224* Initial Action Decl:: Code run before parsing starts.
225* Destructor Decl:: Declaring how symbols are freed.
226* Expect Decl:: Suppressing warnings about parsing conflicts.
227* Start Decl:: Specifying the start symbol.
228* Pure Decl:: Requesting a reentrant parser.
229* Push Decl:: Requesting a push parser.
230* Decl Summary:: Table of all Bison declarations.
231
232Parser C-Language Interface
233
234* Parser Function:: How to call @code{yyparse} and what it returns.
235* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
236* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
237* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
238* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
239* Lexical:: You must supply a function @code{yylex}
240 which reads tokens.
241* Error Reporting:: You must supply a function @code{yyerror}.
242* Action Features:: Special features for use in actions.
243* Internationalization:: How to let the parser speak in the user's
244 native language.
245
246The Lexical Analyzer Function @code{yylex}
247
248* Calling Convention:: How @code{yyparse} calls @code{yylex}.
249* Token Values:: How @code{yylex} must return the semantic value
250 of the token it has read.
251* Token Locations:: How @code{yylex} must return the text location
252 (line number, etc.) of the token, if the
253 actions want that.
254* Pure Calling:: How the calling convention differs in a pure parser
255 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
256
257The Bison Parser Algorithm
258
259* Lookahead:: Parser looks one token ahead when deciding what to do.
260* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
261* Precedence:: Operator precedence works by resolving conflicts.
262* Contextual Precedence:: When an operator's precedence depends on context.
263* Parser States:: The parser is a finite-state-machine with stack.
264* Reduce/Reduce:: When two rules are applicable in the same situation.
265* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
266* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
267* Memory Management:: What happens when memory is exhausted. How to avoid it.
268
269Operator Precedence
270
271* Why Precedence:: An example showing why precedence is needed.
272* Using Precedence:: How to specify precedence and associativity.
273* Precedence Only:: How to specify precedence only.
274* Precedence Examples:: How these features are used in the previous example.
275* How Precedence:: How they work.
276
277Handling Context Dependencies
278
279* Semantic Tokens:: Token parsing can depend on the semantic context.
280* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
281* Tie-in Recovery:: Lexical tie-ins have implications for how
282 error recovery rules must be written.
283
284Debugging Your Parser
285
286* Understanding:: Understanding the structure of your parser.
287* Tracing:: Tracing the execution of your parser.
288
289Invoking Bison
290
291* Bison Options:: All the options described in detail,
292 in alphabetical order by short options.
293* Option Cross Key:: Alphabetical list of long options.
294* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
295
296Parsers Written In Other Languages
297
298* C++ Parsers:: The interface to generate C++ parser classes
299* Java Parsers:: The interface to generate Java parser classes
300
301C++ Parsers
302
303* C++ Bison Interface:: Asking for C++ parser generation
304* C++ Semantic Values:: %union vs. C++
305* C++ Location Values:: The position and location classes
306* C++ Parser Interface:: Instantiating and running the parser
307* C++ Scanner Interface:: Exchanges between yylex and parse
308* A Complete C++ Example:: Demonstrating their use
309
310A Complete C++ Example
311
312* Calc++ --- C++ Calculator:: The specifications
313* Calc++ Parsing Driver:: An active parsing context
314* Calc++ Parser:: A parser class
315* Calc++ Scanner:: A pure C++ Flex scanner
316* Calc++ Top Level:: Conducting the band
317
318Java Parsers
319
320* Java Bison Interface:: Asking for Java parser generation
321* Java Semantic Values:: %type and %token vs. Java
322* Java Location Values:: The position and location classes
323* Java Parser Interface:: Instantiating and running the parser
324* Java Scanner Interface:: Specifying the scanner for the parser
325* Java Action Features:: Special features for use in actions
326* Java Differences:: Differences between C/C++ and Java Grammars
327* Java Declarations Summary:: List of Bison declarations used with Java
328
329Frequently Asked Questions
330
331* Memory Exhausted:: Breaking the Stack Limits
332* How Can I Reset the Parser:: @code{yyparse} Keeps some State
333* Strings are Destroyed:: @code{yylval} Loses Track of Strings
334* Implementing Gotos/Loops:: Control Flow in the Calculator
335* Multiple start-symbols:: Factoring closely related grammars
336* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
337* I can't build Bison:: Troubleshooting
338* Where can I find help?:: Troubleshouting
339* Bug Reports:: Troublereporting
340* More Languages:: Parsers in C++, Java, and so on
341* Beta Testing:: Experimenting development versions
342* Mailing Lists:: Meeting other Bison users
343
344Copying This Manual
345
346* Copying This Manual:: License for copying this manual.
347
348@end detailmenu
349@end menu
350
351@node Introduction
352@unnumbered Introduction
353@cindex introduction
354
355@dfn{Bison} is a general-purpose parser generator that converts an
356annotated context-free grammar into a deterministic or @acronym{GLR}
357parser employing @acronym{LALR}(1), @acronym{IELR}(1), or canonical
358@acronym{LR}(1) parser tables.
359Once you are proficient with Bison, you can use it to develop a wide
360range of language parsers, from those used in simple desk calculators to
361complex programming languages.
362
363Bison is upward compatible with Yacc: all properly-written Yacc grammars
364ought to work with Bison with no change. Anyone familiar with Yacc
365should be able to use Bison with little trouble. You need to be fluent in
366C or C++ programming in order to use Bison or to understand this manual.
367
368We begin with tutorial chapters that explain the basic concepts of using
369Bison and show three explained examples, each building on the last. If you
370don't know Bison or Yacc, start by reading these chapters. Reference
371chapters follow which describe specific aspects of Bison in detail.
372
373Bison was written primarily by Robert Corbett; Richard Stallman made it
374Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
375multi-character string literals and other features.
376
377This edition corresponds to version @value{VERSION} of Bison.
378
379@node Conditions
380@unnumbered Conditions for Using Bison
381
382The distribution terms for Bison-generated parsers permit using the
383parsers in nonfree programs. Before Bison version 2.2, these extra
384permissions applied only when Bison was generating @acronym{LALR}(1)
385parsers in C@. And before Bison version 1.24, Bison-generated
386parsers could be used only in programs that were free software.
387
388The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
389compiler, have never
390had such a requirement. They could always be used for nonfree
391software. The reason Bison was different was not due to a special
392policy decision; it resulted from applying the usual General Public
393License to all of the Bison source code.
394
395The output of the Bison utility---the Bison parser file---contains a
396verbatim copy of a sizable piece of Bison, which is the code for the
397parser's implementation. (The actions from your grammar are inserted
398into this implementation at one point, but most of the rest of the
399implementation is not changed.) When we applied the @acronym{GPL}
400terms to the skeleton code for the parser's implementation,
401the effect was to restrict the use of Bison output to free software.
402
403We didn't change the terms because of sympathy for people who want to
404make software proprietary. @strong{Software should be free.} But we
405concluded that limiting Bison's use to free software was doing little to
406encourage people to make other software free. So we decided to make the
407practical conditions for using Bison match the practical conditions for
408using the other @acronym{GNU} tools.
409
410This exception applies when Bison is generating code for a parser.
411You can tell whether the exception applies to a Bison output file by
412inspecting the file for text beginning with ``As a special
413exception@dots{}''. The text spells out the exact terms of the
414exception.
415
416@node Copying
417@unnumbered GNU GENERAL PUBLIC LICENSE
418@include gpl-3.0.texi
419
420@node Concepts
421@chapter The Concepts of Bison
422
423This chapter introduces many of the basic concepts without which the
424details of Bison will not make sense. If you do not already know how to
425use Bison or Yacc, we suggest you start by reading this chapter carefully.
426
427@menu
428* Language and Grammar:: Languages and context-free grammars,
429 as mathematical ideas.
430* Grammar in Bison:: How we represent grammars for Bison's sake.
431* Semantic Values:: Each token or syntactic grouping can have
432 a semantic value (the value of an integer,
433 the name of an identifier, etc.).
434* Semantic Actions:: Each rule can have an action containing C code.
435* GLR Parsers:: Writing parsers for general context-free languages.
436* Locations Overview:: Tracking Locations.
437* Bison Parser:: What are Bison's input and output,
438 how is the output used?
439* Stages:: Stages in writing and running Bison grammars.
440* Grammar Layout:: Overall structure of a Bison grammar file.
441@end menu
442
443@node Language and Grammar
444@section Languages and Context-Free Grammars
445
446@cindex context-free grammar
447@cindex grammar, context-free
448In order for Bison to parse a language, it must be described by a
449@dfn{context-free grammar}. This means that you specify one or more
450@dfn{syntactic groupings} and give rules for constructing them from their
451parts. For example, in the C language, one kind of grouping is called an
452`expression'. One rule for making an expression might be, ``An expression
453can be made of a minus sign and another expression''. Another would be,
454``An expression can be an integer''. As you can see, rules are often
455recursive, but there must be at least one rule which leads out of the
456recursion.
457
458@cindex @acronym{BNF}
459@cindex Backus-Naur form
460The most common formal system for presenting such rules for humans to read
461is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
462order to specify the language Algol 60. Any grammar expressed in
463@acronym{BNF} is a context-free grammar. The input to Bison is
464essentially machine-readable @acronym{BNF}.
465
466@cindex @acronym{LALR}(1) grammars
467@cindex @acronym{IELR}(1) grammars
468@cindex @acronym{LR}(1) grammars
469There are various important subclasses of context-free grammars.
470Although it can handle almost all context-free grammars, Bison is
471optimized for what are called @acronym{LR}(1) grammars.
472In brief, in these grammars, it must be possible to tell how to parse
473any portion of an input string with just a single token of lookahead.
474For historical reasons, Bison by default is limited by the additional
475restrictions of @acronym{LALR}(1), which is hard to explain simply.
476@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
477more information on this.
478To escape these additional restrictions, you can request
479@acronym{IELR}(1) or canonical @acronym{LR}(1) parser tables.
480@xref{Decl Summary,,lr.type}, to learn how.
481
482@cindex @acronym{GLR} parsing
483@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
484@cindex ambiguous grammars
485@cindex nondeterministic parsing
486
487Parsers for @acronym{LR}(1) grammars are @dfn{deterministic}, meaning
488roughly that the next grammar rule to apply at any point in the input is
489uniquely determined by the preceding input and a fixed, finite portion
490(called a @dfn{lookahead}) of the remaining input. A context-free
491grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
492apply the grammar rules to get the same inputs. Even unambiguous
493grammars can be @dfn{nondeterministic}, meaning that no fixed
494lookahead always suffices to determine the next grammar rule to apply.
495With the proper declarations, Bison is also able to parse these more
496general context-free grammars, using a technique known as @acronym{GLR}
497parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
498are able to handle any context-free grammar for which the number of
499possible parses of any given string is finite.
500
501@cindex symbols (abstract)
502@cindex token
503@cindex syntactic grouping
504@cindex grouping, syntactic
505In the formal grammatical rules for a language, each kind of syntactic
506unit or grouping is named by a @dfn{symbol}. Those which are built by
507grouping smaller constructs according to grammatical rules are called
508@dfn{nonterminal symbols}; those which can't be subdivided are called
509@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
510corresponding to a single terminal symbol a @dfn{token}, and a piece
511corresponding to a single nonterminal symbol a @dfn{grouping}.
512
513We can use the C language as an example of what symbols, terminal and
514nonterminal, mean. The tokens of C are identifiers, constants (numeric
515and string), and the various keywords, arithmetic operators and
516punctuation marks. So the terminal symbols of a grammar for C include
517`identifier', `number', `string', plus one symbol for each keyword,
518operator or punctuation mark: `if', `return', `const', `static', `int',
519`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
520(These tokens can be subdivided into characters, but that is a matter of
521lexicography, not grammar.)
522
523Here is a simple C function subdivided into tokens:
524
525@ifinfo
526@example
527int /* @r{keyword `int'} */
528square (int x) /* @r{identifier, open-paren, keyword `int',}
529 @r{identifier, close-paren} */
530@{ /* @r{open-brace} */
531 return x * x; /* @r{keyword `return', identifier, asterisk,}
532 @r{identifier, semicolon} */
533@} /* @r{close-brace} */
534@end example
535@end ifinfo
536@ifnotinfo
537@example
538int /* @r{keyword `int'} */
539square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
540@{ /* @r{open-brace} */
541 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
542@} /* @r{close-brace} */
543@end example
544@end ifnotinfo
545
546The syntactic groupings of C include the expression, the statement, the
547declaration, and the function definition. These are represented in the
548grammar of C by nonterminal symbols `expression', `statement',
549`declaration' and `function definition'. The full grammar uses dozens of
550additional language constructs, each with its own nonterminal symbol, in
551order to express the meanings of these four. The example above is a
552function definition; it contains one declaration, and one statement. In
553the statement, each @samp{x} is an expression and so is @samp{x * x}.
554
555Each nonterminal symbol must have grammatical rules showing how it is made
556out of simpler constructs. For example, one kind of C statement is the
557@code{return} statement; this would be described with a grammar rule which
558reads informally as follows:
559
560@quotation
561A `statement' can be made of a `return' keyword, an `expression' and a
562`semicolon'.
563@end quotation
564
565@noindent
566There would be many other rules for `statement', one for each kind of
567statement in C.
568
569@cindex start symbol
570One nonterminal symbol must be distinguished as the special one which
571defines a complete utterance in the language. It is called the @dfn{start
572symbol}. In a compiler, this means a complete input program. In the C
573language, the nonterminal symbol `sequence of definitions and declarations'
574plays this role.
575
576For example, @samp{1 + 2} is a valid C expression---a valid part of a C
577program---but it is not valid as an @emph{entire} C program. In the
578context-free grammar of C, this follows from the fact that `expression' is
579not the start symbol.
580
581The Bison parser reads a sequence of tokens as its input, and groups the
582tokens using the grammar rules. If the input is valid, the end result is
583that the entire token sequence reduces to a single grouping whose symbol is
584the grammar's start symbol. If we use a grammar for C, the entire input
585must be a `sequence of definitions and declarations'. If not, the parser
586reports a syntax error.
587
588@node Grammar in Bison
589@section From Formal Rules to Bison Input
590@cindex Bison grammar
591@cindex grammar, Bison
592@cindex formal grammar
593
594A formal grammar is a mathematical construct. To define the language
595for Bison, you must write a file expressing the grammar in Bison syntax:
596a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
597
598A nonterminal symbol in the formal grammar is represented in Bison input
599as an identifier, like an identifier in C@. By convention, it should be
600in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
601
602The Bison representation for a terminal symbol is also called a @dfn{token
603type}. Token types as well can be represented as C-like identifiers. By
604convention, these identifiers should be upper case to distinguish them from
605nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
606@code{RETURN}. A terminal symbol that stands for a particular keyword in
607the language should be named after that keyword converted to upper case.
608The terminal symbol @code{error} is reserved for error recovery.
609@xref{Symbols}.
610
611A terminal symbol can also be represented as a character literal, just like
612a C character constant. You should do this whenever a token is just a
613single character (parenthesis, plus-sign, etc.): use that same character in
614a literal as the terminal symbol for that token.
615
616A third way to represent a terminal symbol is with a C string constant
617containing several characters. @xref{Symbols}, for more information.
618
619The grammar rules also have an expression in Bison syntax. For example,
620here is the Bison rule for a C @code{return} statement. The semicolon in
621quotes is a literal character token, representing part of the C syntax for
622the statement; the naked semicolon, and the colon, are Bison punctuation
623used in every rule.
624
625@example
626stmt: RETURN expr ';'
627 ;
628@end example
629
630@noindent
631@xref{Rules, ,Syntax of Grammar Rules}.
632
633@node Semantic Values
634@section Semantic Values
635@cindex semantic value
636@cindex value, semantic
637
638A formal grammar selects tokens only by their classifications: for example,
639if a rule mentions the terminal symbol `integer constant', it means that
640@emph{any} integer constant is grammatically valid in that position. The
641precise value of the constant is irrelevant to how to parse the input: if
642@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
643grammatical.
644
645But the precise value is very important for what the input means once it is
646parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6473989 as constants in the program! Therefore, each token in a Bison grammar
648has both a token type and a @dfn{semantic value}. @xref{Semantics,
649,Defining Language Semantics},
650for details.
651
652The token type is a terminal symbol defined in the grammar, such as
653@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
654you need to know to decide where the token may validly appear and how to
655group it with other tokens. The grammar rules know nothing about tokens
656except their types.
657
658The semantic value has all the rest of the information about the
659meaning of the token, such as the value of an integer, or the name of an
660identifier. (A token such as @code{','} which is just punctuation doesn't
661need to have any semantic value.)
662
663For example, an input token might be classified as token type
664@code{INTEGER} and have the semantic value 4. Another input token might
665have the same token type @code{INTEGER} but value 3989. When a grammar
666rule says that @code{INTEGER} is allowed, either of these tokens is
667acceptable because each is an @code{INTEGER}. When the parser accepts the
668token, it keeps track of the token's semantic value.
669
670Each grouping can also have a semantic value as well as its nonterminal
671symbol. For example, in a calculator, an expression typically has a
672semantic value that is a number. In a compiler for a programming
673language, an expression typically has a semantic value that is a tree
674structure describing the meaning of the expression.
675
676@node Semantic Actions
677@section Semantic Actions
678@cindex semantic actions
679@cindex actions, semantic
680
681In order to be useful, a program must do more than parse input; it must
682also produce some output based on the input. In a Bison grammar, a grammar
683rule can have an @dfn{action} made up of C statements. Each time the
684parser recognizes a match for that rule, the action is executed.
685@xref{Actions}.
686
687Most of the time, the purpose of an action is to compute the semantic value
688of the whole construct from the semantic values of its parts. For example,
689suppose we have a rule which says an expression can be the sum of two
690expressions. When the parser recognizes such a sum, each of the
691subexpressions has a semantic value which describes how it was built up.
692The action for this rule should create a similar sort of value for the
693newly recognized larger expression.
694
695For example, here is a rule that says an expression can be the sum of
696two subexpressions:
697
698@example
699expr: expr '+' expr @{ $$ = $1 + $3; @}
700 ;
701@end example
702
703@noindent
704The action says how to produce the semantic value of the sum expression
705from the values of the two subexpressions.
706
707@node GLR Parsers
708@section Writing @acronym{GLR} Parsers
709@cindex @acronym{GLR} parsing
710@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
711@findex %glr-parser
712@cindex conflicts
713@cindex shift/reduce conflicts
714@cindex reduce/reduce conflicts
715
716In some grammars, Bison's deterministic
717@acronym{LR}(1) parsing algorithm cannot decide whether to apply a
718certain grammar rule at a given point. That is, it may not be able to
719decide (on the basis of the input read so far) which of two possible
720reductions (applications of a grammar rule) applies, or whether to apply
721a reduction or read more of the input and apply a reduction later in the
722input. These are known respectively as @dfn{reduce/reduce} conflicts
723(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
724(@pxref{Shift/Reduce}).
725
726To use a grammar that is not easily modified to be @acronym{LR}(1), a
727more general parsing algorithm is sometimes necessary. If you include
728@code{%glr-parser} among the Bison declarations in your file
729(@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
730(@acronym{GLR}) parser. These parsers handle Bison grammars that
731contain no unresolved conflicts (i.e., after applying precedence
732declarations) identically to deterministic parsers. However, when
733faced with unresolved shift/reduce and reduce/reduce conflicts,
734@acronym{GLR} parsers use the simple expedient of doing both,
735effectively cloning the parser to follow both possibilities. Each of
736the resulting parsers can again split, so that at any given time, there
737can be any number of possible parses being explored. The parsers
738proceed in lockstep; that is, all of them consume (shift) a given input
739symbol before any of them proceed to the next. Each of the cloned
740parsers eventually meets one of two possible fates: either it runs into
741a parsing error, in which case it simply vanishes, or it merges with
742another parser, because the two of them have reduced the input to an
743identical set of symbols.
744
745During the time that there are multiple parsers, semantic actions are
746recorded, but not performed. When a parser disappears, its recorded
747semantic actions disappear as well, and are never performed. When a
748reduction makes two parsers identical, causing them to merge, Bison
749records both sets of semantic actions. Whenever the last two parsers
750merge, reverting to the single-parser case, Bison resolves all the
751outstanding actions either by precedences given to the grammar rules
752involved, or by performing both actions, and then calling a designated
753user-defined function on the resulting values to produce an arbitrary
754merged result.
755
756@menu
757* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
758* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
759* GLR Semantic Actions:: Deferred semantic actions have special concerns.
760* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
761@end menu
762
763@node Simple GLR Parsers
764@subsection Using @acronym{GLR} on Unambiguous Grammars
765@cindex @acronym{GLR} parsing, unambiguous grammars
766@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
767@findex %glr-parser
768@findex %expect-rr
769@cindex conflicts
770@cindex reduce/reduce conflicts
771@cindex shift/reduce conflicts
772
773In the simplest cases, you can use the @acronym{GLR} algorithm
774to parse grammars that are unambiguous but fail to be @acronym{LR}(1).
775Such grammars typically require more than one symbol of lookahead.
776
777Consider a problem that
778arises in the declaration of enumerated and subrange types in the
779programming language Pascal. Here are some examples:
780
781@example
782type subrange = lo .. hi;
783type enum = (a, b, c);
784@end example
785
786@noindent
787The original language standard allows only numeric
788literals and constant identifiers for the subrange bounds (@samp{lo}
789and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
79010206) and many other
791Pascal implementations allow arbitrary expressions there. This gives
792rise to the following situation, containing a superfluous pair of
793parentheses:
794
795@example
796type subrange = (a) .. b;
797@end example
798
799@noindent
800Compare this to the following declaration of an enumerated
801type with only one value:
802
803@example
804type enum = (a);
805@end example
806
807@noindent
808(These declarations are contrived, but they are syntactically
809valid, and more-complicated cases can come up in practical programs.)
810
811These two declarations look identical until the @samp{..} token.
812With normal @acronym{LR}(1) one-token lookahead it is not
813possible to decide between the two forms when the identifier
814@samp{a} is parsed. It is, however, desirable
815for a parser to decide this, since in the latter case
816@samp{a} must become a new identifier to represent the enumeration
817value, while in the former case @samp{a} must be evaluated with its
818current meaning, which may be a constant or even a function call.
819
820You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
821to be resolved later, but this typically requires substantial
822contortions in both semantic actions and large parts of the
823grammar, where the parentheses are nested in the recursive rules for
824expressions.
825
826You might think of using the lexer to distinguish between the two
827forms by returning different tokens for currently defined and
828undefined identifiers. But if these declarations occur in a local
829scope, and @samp{a} is defined in an outer scope, then both forms
830are possible---either locally redefining @samp{a}, or using the
831value of @samp{a} from the outer scope. So this approach cannot
832work.
833
834A simple solution to this problem is to declare the parser to
835use the @acronym{GLR} algorithm.
836When the @acronym{GLR} parser reaches the critical state, it
837merely splits into two branches and pursues both syntax rules
838simultaneously. Sooner or later, one of them runs into a parsing
839error. If there is a @samp{..} token before the next
840@samp{;}, the rule for enumerated types fails since it cannot
841accept @samp{..} anywhere; otherwise, the subrange type rule
842fails since it requires a @samp{..} token. So one of the branches
843fails silently, and the other one continues normally, performing
844all the intermediate actions that were postponed during the split.
845
846If the input is syntactically incorrect, both branches fail and the parser
847reports a syntax error as usual.
848
849The effect of all this is that the parser seems to ``guess'' the
850correct branch to take, or in other words, it seems to use more
851lookahead than the underlying @acronym{LR}(1) algorithm actually allows
852for. In this example, @acronym{LR}(2) would suffice, but also some cases
853that are not @acronym{LR}(@math{k}) for any @math{k} can be handled this way.
854
855In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
856and the current Bison parser even takes exponential time and space
857for some grammars. In practice, this rarely happens, and for many
858grammars it is possible to prove that it cannot happen.
859The present example contains only one conflict between two
860rules, and the type-declaration context containing the conflict
861cannot be nested. So the number of
862branches that can exist at any time is limited by the constant 2,
863and the parsing time is still linear.
864
865Here is a Bison grammar corresponding to the example above. It
866parses a vastly simplified form of Pascal type declarations.
867
868@example
869%token TYPE DOTDOT ID
870
871@group
872%left '+' '-'
873%left '*' '/'
874@end group
875
876%%
877
878@group
879type_decl : TYPE ID '=' type ';'
880 ;
881@end group
882
883@group
884type : '(' id_list ')'
885 | expr DOTDOT expr
886 ;
887@end group
888
889@group
890id_list : ID
891 | id_list ',' ID
892 ;
893@end group
894
895@group
896expr : '(' expr ')'
897 | expr '+' expr
898 | expr '-' expr
899 | expr '*' expr
900 | expr '/' expr
901 | ID
902 ;
903@end group
904@end example
905
906When used as a normal @acronym{LR}(1) grammar, Bison correctly complains
907about one reduce/reduce conflict. In the conflicting situation the
908parser chooses one of the alternatives, arbitrarily the one
909declared first. Therefore the following correct input is not
910recognized:
911
912@example
913type t = (a) .. b;
914@end example
915
916The parser can be turned into a @acronym{GLR} parser, while also telling Bison
917to be silent about the one known reduce/reduce conflict, by
918adding these two declarations to the Bison input file (before the first
919@samp{%%}):
920
921@example
922%glr-parser
923%expect-rr 1
924@end example
925
926@noindent
927No change in the grammar itself is required. Now the
928parser recognizes all valid declarations, according to the
929limited syntax above, transparently. In fact, the user does not even
930notice when the parser splits.
931
932So here we have a case where we can use the benefits of @acronym{GLR},
933almost without disadvantages. Even in simple cases like this, however,
934there are at least two potential problems to beware. First, always
935analyze the conflicts reported by Bison to make sure that @acronym{GLR}
936splitting is only done where it is intended. A @acronym{GLR} parser
937splitting inadvertently may cause problems less obvious than an
938@acronym{LR} parser statically choosing the wrong alternative in a
939conflict. Second, consider interactions with the lexer (@pxref{Semantic
940Tokens}) with great care. Since a split parser consumes tokens without
941performing any actions during the split, the lexer cannot obtain
942information via parser actions. Some cases of lexer interactions can be
943eliminated by using @acronym{GLR} to shift the complications from the
944lexer to the parser. You must check the remaining cases for
945correctness.
946
947In our example, it would be safe for the lexer to return tokens based on
948their current meanings in some symbol table, because no new symbols are
949defined in the middle of a type declaration. Though it is possible for
950a parser to define the enumeration constants as they are parsed, before
951the type declaration is completed, it actually makes no difference since
952they cannot be used within the same enumerated type declaration.
953
954@node Merging GLR Parses
955@subsection Using @acronym{GLR} to Resolve Ambiguities
956@cindex @acronym{GLR} parsing, ambiguous grammars
957@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
958@findex %dprec
959@findex %merge
960@cindex conflicts
961@cindex reduce/reduce conflicts
962
963Let's consider an example, vastly simplified from a C++ grammar.
964
965@example
966%@{
967 #include <stdio.h>
968 #define YYSTYPE char const *
969 int yylex (void);
970 void yyerror (char const *);
971%@}
972
973%token TYPENAME ID
974
975%right '='
976%left '+'
977
978%glr-parser
979
980%%
981
982prog :
983 | prog stmt @{ printf ("\n"); @}
984 ;
985
986stmt : expr ';' %dprec 1
987 | decl %dprec 2
988 ;
989
990expr : ID @{ printf ("%s ", $$); @}
991 | TYPENAME '(' expr ')'
992 @{ printf ("%s <cast> ", $1); @}
993 | expr '+' expr @{ printf ("+ "); @}
994 | expr '=' expr @{ printf ("= "); @}
995 ;
996
997decl : TYPENAME declarator ';'
998 @{ printf ("%s <declare> ", $1); @}
999 | TYPENAME declarator '=' expr ';'
1000 @{ printf ("%s <init-declare> ", $1); @}
1001 ;
1002
1003declarator : ID @{ printf ("\"%s\" ", $1); @}
1004 | '(' declarator ')'
1005 ;
1006@end example
1007
1008@noindent
1009This models a problematic part of the C++ grammar---the ambiguity between
1010certain declarations and statements. For example,
1011
1012@example
1013T (x) = y+z;
1014@end example
1015
1016@noindent
1017parses as either an @code{expr} or a @code{stmt}
1018(assuming that @samp{T} is recognized as a @code{TYPENAME} and
1019@samp{x} as an @code{ID}).
1020Bison detects this as a reduce/reduce conflict between the rules
1021@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
1022time it encounters @code{x} in the example above. Since this is a
1023@acronym{GLR} parser, it therefore splits the problem into two parses, one for
1024each choice of resolving the reduce/reduce conflict.
1025Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1026however, neither of these parses ``dies,'' because the grammar as it stands is
1027ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1028the other reduces @code{stmt : decl}, after which both parsers are in an
1029identical state: they've seen @samp{prog stmt} and have the same unprocessed
1030input remaining. We say that these parses have @dfn{merged.}
1031
1032At this point, the @acronym{GLR} parser requires a specification in the
1033grammar of how to choose between the competing parses.
1034In the example above, the two @code{%dprec}
1035declarations specify that Bison is to give precedence
1036to the parse that interprets the example as a
1037@code{decl}, which implies that @code{x} is a declarator.
1038The parser therefore prints
1039
1040@example
1041"x" y z + T <init-declare>
1042@end example
1043
1044The @code{%dprec} declarations only come into play when more than one
1045parse survives. Consider a different input string for this parser:
1046
1047@example
1048T (x) + y;
1049@end example
1050
1051@noindent
1052This is another example of using @acronym{GLR} to parse an unambiguous
1053construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
1054Here, there is no ambiguity (this cannot be parsed as a declaration).
1055However, at the time the Bison parser encounters @code{x}, it does not
1056have enough information to resolve the reduce/reduce conflict (again,
1057between @code{x} as an @code{expr} or a @code{declarator}). In this
1058case, no precedence declaration is used. Again, the parser splits
1059into two, one assuming that @code{x} is an @code{expr}, and the other
1060assuming @code{x} is a @code{declarator}. The second of these parsers
1061then vanishes when it sees @code{+}, and the parser prints
1062
1063@example
1064x T <cast> y +
1065@end example
1066
1067Suppose that instead of resolving the ambiguity, you wanted to see all
1068the possibilities. For this purpose, you must merge the semantic
1069actions of the two possible parsers, rather than choosing one over the
1070other. To do so, you could change the declaration of @code{stmt} as
1071follows:
1072
1073@example
1074stmt : expr ';' %merge <stmtMerge>
1075 | decl %merge <stmtMerge>
1076 ;
1077@end example
1078
1079@noindent
1080and define the @code{stmtMerge} function as:
1081
1082@example
1083static YYSTYPE
1084stmtMerge (YYSTYPE x0, YYSTYPE x1)
1085@{
1086 printf ("<OR> ");
1087 return "";
1088@}
1089@end example
1090
1091@noindent
1092with an accompanying forward declaration
1093in the C declarations at the beginning of the file:
1094
1095@example
1096%@{
1097 #define YYSTYPE char const *
1098 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1099%@}
1100@end example
1101
1102@noindent
1103With these declarations, the resulting parser parses the first example
1104as both an @code{expr} and a @code{decl}, and prints
1105
1106@example
1107"x" y z + T <init-declare> x T <cast> y z + = <OR>
1108@end example
1109
1110Bison requires that all of the
1111productions that participate in any particular merge have identical
1112@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1113and the parser will report an error during any parse that results in
1114the offending merge.
1115
1116@node GLR Semantic Actions
1117@subsection GLR Semantic Actions
1118
1119@cindex deferred semantic actions
1120By definition, a deferred semantic action is not performed at the same time as
1121the associated reduction.
1122This raises caveats for several Bison features you might use in a semantic
1123action in a @acronym{GLR} parser.
1124
1125@vindex yychar
1126@cindex @acronym{GLR} parsers and @code{yychar}
1127@vindex yylval
1128@cindex @acronym{GLR} parsers and @code{yylval}
1129@vindex yylloc
1130@cindex @acronym{GLR} parsers and @code{yylloc}
1131In any semantic action, you can examine @code{yychar} to determine the type of
1132the lookahead token present at the time of the associated reduction.
1133After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1134you can then examine @code{yylval} and @code{yylloc} to determine the
1135lookahead token's semantic value and location, if any.
1136In a nondeferred semantic action, you can also modify any of these variables to
1137influence syntax analysis.
1138@xref{Lookahead, ,Lookahead Tokens}.
1139
1140@findex yyclearin
1141@cindex @acronym{GLR} parsers and @code{yyclearin}
1142In a deferred semantic action, it's too late to influence syntax analysis.
1143In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1144shallow copies of the values they had at the time of the associated reduction.
1145For this reason alone, modifying them is dangerous.
1146Moreover, the result of modifying them is undefined and subject to change with
1147future versions of Bison.
1148For example, if a semantic action might be deferred, you should never write it
1149to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1150memory referenced by @code{yylval}.
1151
1152@findex YYERROR
1153@cindex @acronym{GLR} parsers and @code{YYERROR}
1154Another Bison feature requiring special consideration is @code{YYERROR}
1155(@pxref{Action Features}), which you can invoke in a semantic action to
1156initiate error recovery.
1157During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is
1158the same as its effect in a deterministic parser.
1159In a deferred semantic action, its effect is undefined.
1160@c The effect is probably a syntax error at the split point.
1161
1162Also, see @ref{Location Default Action, ,Default Action for Locations}, which
1163describes a special usage of @code{YYLLOC_DEFAULT} in @acronym{GLR} parsers.
1164
1165@node Compiler Requirements
1166@subsection Considerations when Compiling @acronym{GLR} Parsers
1167@cindex @code{inline}
1168@cindex @acronym{GLR} parsers and @code{inline}
1169
1170The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1171later. In addition, they use the @code{inline} keyword, which is not
1172C89, but is C99 and is a common extension in pre-C99 compilers. It is
1173up to the user of these parsers to handle
1174portability issues. For instance, if using Autoconf and the Autoconf
1175macro @code{AC_C_INLINE}, a mere
1176
1177@example
1178%@{
1179 #include <config.h>
1180%@}
1181@end example
1182
1183@noindent
1184will suffice. Otherwise, we suggest
1185
1186@example
1187%@{
1188 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1189 #define inline
1190 #endif
1191%@}
1192@end example
1193
1194@node Locations Overview
1195@section Locations
1196@cindex location
1197@cindex textual location
1198@cindex location, textual
1199
1200Many applications, like interpreters or compilers, have to produce verbose
1201and useful error messages. To achieve this, one must be able to keep track of
1202the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
1203Bison provides a mechanism for handling these locations.
1204
1205Each token has a semantic value. In a similar fashion, each token has an
1206associated location, but the type of locations is the same for all tokens and
1207groupings. Moreover, the output parser is equipped with a default data
1208structure for storing locations (@pxref{Locations}, for more details).
1209
1210Like semantic values, locations can be reached in actions using a dedicated
1211set of constructs. In the example above, the location of the whole grouping
1212is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1213@code{@@3}.
1214
1215When a rule is matched, a default action is used to compute the semantic value
1216of its left hand side (@pxref{Actions}). In the same way, another default
1217action is used for locations. However, the action for locations is general
1218enough for most cases, meaning there is usually no need to describe for each
1219rule how @code{@@$} should be formed. When building a new location for a given
1220grouping, the default behavior of the output parser is to take the beginning
1221of the first symbol, and the end of the last symbol.
1222
1223@node Bison Parser
1224@section Bison Output: the Parser File
1225@cindex Bison parser
1226@cindex Bison utility
1227@cindex lexical analyzer, purpose
1228@cindex parser
1229
1230When you run Bison, you give it a Bison grammar file as input. The output
1231is a C source file that parses the language described by the grammar.
1232This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1233utility and the Bison parser are two distinct programs: the Bison utility
1234is a program whose output is the Bison parser that becomes part of your
1235program.
1236
1237The job of the Bison parser is to group tokens into groupings according to
1238the grammar rules---for example, to build identifiers and operators into
1239expressions. As it does this, it runs the actions for the grammar rules it
1240uses.
1241
1242The tokens come from a function called the @dfn{lexical analyzer} that
1243you must supply in some fashion (such as by writing it in C). The Bison
1244parser calls the lexical analyzer each time it wants a new token. It
1245doesn't know what is ``inside'' the tokens (though their semantic values
1246may reflect this). Typically the lexical analyzer makes the tokens by
1247parsing characters of text, but Bison does not depend on this.
1248@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
1249
1250The Bison parser file is C code which defines a function named
1251@code{yyparse} which implements that grammar. This function does not make
1252a complete C program: you must supply some additional functions. One is
1253the lexical analyzer. Another is an error-reporting function which the
1254parser calls to report an error. In addition, a complete C program must
1255start with a function called @code{main}; you have to provide this, and
1256arrange for it to call @code{yyparse} or the parser will never run.
1257@xref{Interface, ,Parser C-Language Interface}.
1258
1259Aside from the token type names and the symbols in the actions you
1260write, all symbols defined in the Bison parser file itself
1261begin with @samp{yy} or @samp{YY}. This includes interface functions
1262such as the lexical analyzer function @code{yylex}, the error reporting
1263function @code{yyerror} and the parser function @code{yyparse} itself.
1264This also includes numerous identifiers used for internal purposes.
1265Therefore, you should avoid using C identifiers starting with @samp{yy}
1266or @samp{YY} in the Bison grammar file except for the ones defined in
1267this manual. Also, you should avoid using the C identifiers
1268@samp{malloc} and @samp{free} for anything other than their usual
1269meanings.
1270
1271In some cases the Bison parser file includes system headers, and in
1272those cases your code should respect the identifiers reserved by those
1273headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
1274@code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
1275declare memory allocators and related types. @code{<libintl.h>} is
1276included if message translation is in use
1277(@pxref{Internationalization}). Other system headers may
1278be included if you define @code{YYDEBUG} to a nonzero value
1279(@pxref{Tracing, ,Tracing Your Parser}).
1280
1281@node Stages
1282@section Stages in Using Bison
1283@cindex stages in using Bison
1284@cindex using Bison
1285
1286The actual language-design process using Bison, from grammar specification
1287to a working compiler or interpreter, has these parts:
1288
1289@enumerate
1290@item
1291Formally specify the grammar in a form recognized by Bison
1292(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1293in the language, describe the action that is to be taken when an
1294instance of that rule is recognized. The action is described by a
1295sequence of C statements.
1296
1297@item
1298Write a lexical analyzer to process input and pass tokens to the parser.
1299The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1300Lexical Analyzer Function @code{yylex}}). It could also be produced
1301using Lex, but the use of Lex is not discussed in this manual.
1302
1303@item
1304Write a controlling function that calls the Bison-produced parser.
1305
1306@item
1307Write error-reporting routines.
1308@end enumerate
1309
1310To turn this source code as written into a runnable program, you
1311must follow these steps:
1312
1313@enumerate
1314@item
1315Run Bison on the grammar to produce the parser.
1316
1317@item
1318Compile the code output by Bison, as well as any other source files.
1319
1320@item
1321Link the object files to produce the finished product.
1322@end enumerate
1323
1324@node Grammar Layout
1325@section The Overall Layout of a Bison Grammar
1326@cindex grammar file
1327@cindex file format
1328@cindex format of grammar file
1329@cindex layout of Bison grammar
1330
1331The input file for the Bison utility is a @dfn{Bison grammar file}. The
1332general form of a Bison grammar file is as follows:
1333
1334@example
1335%@{
1336@var{Prologue}
1337%@}
1338
1339@var{Bison declarations}
1340
1341%%
1342@var{Grammar rules}
1343%%
1344@var{Epilogue}
1345@end example
1346
1347@noindent
1348The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1349in every Bison grammar file to separate the sections.
1350
1351The prologue may define types and variables used in the actions. You can
1352also use preprocessor commands to define macros used there, and use
1353@code{#include} to include header files that do any of these things.
1354You need to declare the lexical analyzer @code{yylex} and the error
1355printer @code{yyerror} here, along with any other global identifiers
1356used by the actions in the grammar rules.
1357
1358The Bison declarations declare the names of the terminal and nonterminal
1359symbols, and may also describe operator precedence and the data types of
1360semantic values of various symbols.
1361
1362The grammar rules define how to construct each nonterminal symbol from its
1363parts.
1364
1365The epilogue can contain any code you want to use. Often the
1366definitions of functions declared in the prologue go here. In a
1367simple program, all the rest of the program can go here.
1368
1369@node Examples
1370@chapter Examples
1371@cindex simple examples
1372@cindex examples, simple
1373
1374Now we show and explain three sample programs written using Bison: a
1375reverse polish notation calculator, an algebraic (infix) notation
1376calculator, and a multi-function calculator. All three have been tested
1377under BSD Unix 4.3; each produces a usable, though limited, interactive
1378desk-top calculator.
1379
1380These examples are simple, but Bison grammars for real programming
1381languages are written the same way. You can copy these examples into a
1382source file to try them.
1383
1384@menu
1385* RPN Calc:: Reverse polish notation calculator;
1386 a first example with no operator precedence.
1387* Infix Calc:: Infix (algebraic) notation calculator.
1388 Operator precedence is introduced.
1389* Simple Error Recovery:: Continuing after syntax errors.
1390* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
1391* Multi-function Calc:: Calculator with memory and trig functions.
1392 It uses multiple data-types for semantic values.
1393* Exercises:: Ideas for improving the multi-function calculator.
1394@end menu
1395
1396@node RPN Calc
1397@section Reverse Polish Notation Calculator
1398@cindex reverse polish notation
1399@cindex polish notation calculator
1400@cindex @code{rpcalc}
1401@cindex calculator, simple
1402
1403The first example is that of a simple double-precision @dfn{reverse polish
1404notation} calculator (a calculator using postfix operators). This example
1405provides a good starting point, since operator precedence is not an issue.
1406The second example will illustrate how operator precedence is handled.
1407
1408The source code for this calculator is named @file{rpcalc.y}. The
1409@samp{.y} extension is a convention used for Bison input files.
1410
1411@menu
1412* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1413* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1414* Rpcalc Lexer:: The lexical analyzer.
1415* Rpcalc Main:: The controlling function.
1416* Rpcalc Error:: The error reporting function.
1417* Rpcalc Generate:: Running Bison on the grammar file.
1418* Rpcalc Compile:: Run the C compiler on the output code.
1419@end menu
1420
1421@node Rpcalc Declarations
1422@subsection Declarations for @code{rpcalc}
1423
1424Here are the C and Bison declarations for the reverse polish notation
1425calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1426
1427@example
1428/* Reverse polish notation calculator. */
1429
1430%@{
1431 #define YYSTYPE double
1432 #include <math.h>
1433 int yylex (void);
1434 void yyerror (char const *);
1435%@}
1436
1437%token NUM
1438
1439%% /* Grammar rules and actions follow. */
1440@end example
1441
1442The declarations section (@pxref{Prologue, , The prologue}) contains two
1443preprocessor directives and two forward declarations.
1444
1445The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1446specifying the C data type for semantic values of both tokens and
1447groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1448Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1449don't define it, @code{int} is the default. Because we specify
1450@code{double}, each token and each expression has an associated value,
1451which is a floating point number.
1452
1453The @code{#include} directive is used to declare the exponentiation
1454function @code{pow}.
1455
1456The forward declarations for @code{yylex} and @code{yyerror} are
1457needed because the C language requires that functions be declared
1458before they are used. These functions will be defined in the
1459epilogue, but the parser calls them so they must be declared in the
1460prologue.
1461
1462The second section, Bison declarations, provides information to Bison
1463about the token types (@pxref{Bison Declarations, ,The Bison
1464Declarations Section}). Each terminal symbol that is not a
1465single-character literal must be declared here. (Single-character
1466literals normally don't need to be declared.) In this example, all the
1467arithmetic operators are designated by single-character literals, so the
1468only terminal symbol that needs to be declared is @code{NUM}, the token
1469type for numeric constants.
1470
1471@node Rpcalc Rules
1472@subsection Grammar Rules for @code{rpcalc}
1473
1474Here are the grammar rules for the reverse polish notation calculator.
1475
1476@example
1477input: /* empty */
1478 | input line
1479;
1480
1481line: '\n'
1482 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1483;
1484
1485exp: NUM @{ $$ = $1; @}
1486 | exp exp '+' @{ $$ = $1 + $2; @}
1487 | exp exp '-' @{ $$ = $1 - $2; @}
1488 | exp exp '*' @{ $$ = $1 * $2; @}
1489 | exp exp '/' @{ $$ = $1 / $2; @}
1490 /* Exponentiation */
1491 | exp exp '^' @{ $$ = pow ($1, $2); @}
1492 /* Unary minus */
1493 | exp 'n' @{ $$ = -$1; @}
1494;
1495%%
1496@end example
1497
1498The groupings of the rpcalc ``language'' defined here are the expression
1499(given the name @code{exp}), the line of input (@code{line}), and the
1500complete input transcript (@code{input}). Each of these nonterminal
1501symbols has several alternate rules, joined by the vertical bar @samp{|}
1502which is read as ``or''. The following sections explain what these rules
1503mean.
1504
1505The semantics of the language is determined by the actions taken when a
1506grouping is recognized. The actions are the C code that appears inside
1507braces. @xref{Actions}.
1508
1509You must specify these actions in C, but Bison provides the means for
1510passing semantic values between the rules. In each action, the
1511pseudo-variable @code{$$} stands for the semantic value for the grouping
1512that the rule is going to construct. Assigning a value to @code{$$} is the
1513main job of most actions. The semantic values of the components of the
1514rule are referred to as @code{$1}, @code{$2}, and so on.
1515
1516@menu
1517* Rpcalc Input::
1518* Rpcalc Line::
1519* Rpcalc Expr::
1520@end menu
1521
1522@node Rpcalc Input
1523@subsubsection Explanation of @code{input}
1524
1525Consider the definition of @code{input}:
1526
1527@example
1528input: /* empty */
1529 | input line
1530;
1531@end example
1532
1533This definition reads as follows: ``A complete input is either an empty
1534string, or a complete input followed by an input line''. Notice that
1535``complete input'' is defined in terms of itself. This definition is said
1536to be @dfn{left recursive} since @code{input} appears always as the
1537leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1538
1539The first alternative is empty because there are no symbols between the
1540colon and the first @samp{|}; this means that @code{input} can match an
1541empty string of input (no tokens). We write the rules this way because it
1542is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1543It's conventional to put an empty alternative first and write the comment
1544@samp{/* empty */} in it.
1545
1546The second alternate rule (@code{input line}) handles all nontrivial input.
1547It means, ``After reading any number of lines, read one more line if
1548possible.'' The left recursion makes this rule into a loop. Since the
1549first alternative matches empty input, the loop can be executed zero or
1550more times.
1551
1552The parser function @code{yyparse} continues to process input until a
1553grammatical error is seen or the lexical analyzer says there are no more
1554input tokens; we will arrange for the latter to happen at end-of-input.
1555
1556@node Rpcalc Line
1557@subsubsection Explanation of @code{line}
1558
1559Now consider the definition of @code{line}:
1560
1561@example
1562line: '\n'
1563 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1564;
1565@end example
1566
1567The first alternative is a token which is a newline character; this means
1568that rpcalc accepts a blank line (and ignores it, since there is no
1569action). The second alternative is an expression followed by a newline.
1570This is the alternative that makes rpcalc useful. The semantic value of
1571the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1572question is the first symbol in the alternative. The action prints this
1573value, which is the result of the computation the user asked for.
1574
1575This action is unusual because it does not assign a value to @code{$$}. As
1576a consequence, the semantic value associated with the @code{line} is
1577uninitialized (its value will be unpredictable). This would be a bug if
1578that value were ever used, but we don't use it: once rpcalc has printed the
1579value of the user's input line, that value is no longer needed.
1580
1581@node Rpcalc Expr
1582@subsubsection Explanation of @code{expr}
1583
1584The @code{exp} grouping has several rules, one for each kind of expression.
1585The first rule handles the simplest expressions: those that are just numbers.
1586The second handles an addition-expression, which looks like two expressions
1587followed by a plus-sign. The third handles subtraction, and so on.
1588
1589@example
1590exp: NUM
1591 | exp exp '+' @{ $$ = $1 + $2; @}
1592 | exp exp '-' @{ $$ = $1 - $2; @}
1593 @dots{}
1594 ;
1595@end example
1596
1597We have used @samp{|} to join all the rules for @code{exp}, but we could
1598equally well have written them separately:
1599
1600@example
1601exp: NUM ;
1602exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1603exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1604 @dots{}
1605@end example
1606
1607Most of the rules have actions that compute the value of the expression in
1608terms of the value of its parts. For example, in the rule for addition,
1609@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1610the second one. The third component, @code{'+'}, has no meaningful
1611associated semantic value, but if it had one you could refer to it as
1612@code{$3}. When @code{yyparse} recognizes a sum expression using this
1613rule, the sum of the two subexpressions' values is produced as the value of
1614the entire expression. @xref{Actions}.
1615
1616You don't have to give an action for every rule. When a rule has no
1617action, Bison by default copies the value of @code{$1} into @code{$$}.
1618This is what happens in the first rule (the one that uses @code{NUM}).
1619
1620The formatting shown here is the recommended convention, but Bison does
1621not require it. You can add or change white space as much as you wish.
1622For example, this:
1623
1624@example
1625exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
1626@end example
1627
1628@noindent
1629means the same thing as this:
1630
1631@example
1632exp: NUM
1633 | exp exp '+' @{ $$ = $1 + $2; @}
1634 | @dots{}
1635;
1636@end example
1637
1638@noindent
1639The latter, however, is much more readable.
1640
1641@node Rpcalc Lexer
1642@subsection The @code{rpcalc} Lexical Analyzer
1643@cindex writing a lexical analyzer
1644@cindex lexical analyzer, writing
1645
1646The lexical analyzer's job is low-level parsing: converting characters
1647or sequences of characters into tokens. The Bison parser gets its
1648tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1649Analyzer Function @code{yylex}}.
1650
1651Only a simple lexical analyzer is needed for the @acronym{RPN}
1652calculator. This
1653lexical analyzer skips blanks and tabs, then reads in numbers as
1654@code{double} and returns them as @code{NUM} tokens. Any other character
1655that isn't part of a number is a separate token. Note that the token-code
1656for such a single-character token is the character itself.
1657
1658The return value of the lexical analyzer function is a numeric code which
1659represents a token type. The same text used in Bison rules to stand for
1660this token type is also a C expression for the numeric code for the type.
1661This works in two ways. If the token type is a character literal, then its
1662numeric code is that of the character; you can use the same
1663character literal in the lexical analyzer to express the number. If the
1664token type is an identifier, that identifier is defined by Bison as a C
1665macro whose definition is the appropriate number. In this example,
1666therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1667
1668The semantic value of the token (if it has one) is stored into the
1669global variable @code{yylval}, which is where the Bison parser will look
1670for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1671defined at the beginning of the grammar; @pxref{Rpcalc Declarations,
1672,Declarations for @code{rpcalc}}.)
1673
1674A token type code of zero is returned if the end-of-input is encountered.
1675(Bison recognizes any nonpositive value as indicating end-of-input.)
1676
1677Here is the code for the lexical analyzer:
1678
1679@example
1680@group
1681/* The lexical analyzer returns a double floating point
1682 number on the stack and the token NUM, or the numeric code
1683 of the character read if not a number. It skips all blanks
1684 and tabs, and returns 0 for end-of-input. */
1685
1686#include <ctype.h>
1687@end group
1688
1689@group
1690int
1691yylex (void)
1692@{
1693 int c;
1694
1695 /* Skip white space. */
1696 while ((c = getchar ()) == ' ' || c == '\t')
1697 ;
1698@end group
1699@group
1700 /* Process numbers. */
1701 if (c == '.' || isdigit (c))
1702 @{
1703 ungetc (c, stdin);
1704 scanf ("%lf", &yylval);
1705 return NUM;
1706 @}
1707@end group
1708@group
1709 /* Return end-of-input. */
1710 if (c == EOF)
1711 return 0;
1712 /* Return a single char. */
1713 return c;
1714@}
1715@end group
1716@end example
1717
1718@node Rpcalc Main
1719@subsection The Controlling Function
1720@cindex controlling function
1721@cindex main function in simple example
1722
1723In keeping with the spirit of this example, the controlling function is
1724kept to the bare minimum. The only requirement is that it call
1725@code{yyparse} to start the process of parsing.
1726
1727@example
1728@group
1729int
1730main (void)
1731@{
1732 return yyparse ();
1733@}
1734@end group
1735@end example
1736
1737@node Rpcalc Error
1738@subsection The Error Reporting Routine
1739@cindex error reporting routine
1740
1741When @code{yyparse} detects a syntax error, it calls the error reporting
1742function @code{yyerror} to print an error message (usually but not
1743always @code{"syntax error"}). It is up to the programmer to supply
1744@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1745here is the definition we will use:
1746
1747@example
1748@group
1749#include <stdio.h>
1750
1751/* Called by yyparse on error. */
1752void
1753yyerror (char const *s)
1754@{
1755 fprintf (stderr, "%s\n", s);
1756@}
1757@end group
1758@end example
1759
1760After @code{yyerror} returns, the Bison parser may recover from the error
1761and continue parsing if the grammar contains a suitable error rule
1762(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1763have not written any error rules in this example, so any invalid input will
1764cause the calculator program to exit. This is not clean behavior for a
1765real calculator, but it is adequate for the first example.
1766
1767@node Rpcalc Generate
1768@subsection Running Bison to Make the Parser
1769@cindex running Bison (introduction)
1770
1771Before running Bison to produce a parser, we need to decide how to
1772arrange all the source code in one or more source files. For such a
1773simple example, the easiest thing is to put everything in one file. The
1774definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
1775end, in the epilogue of the file
1776(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
1777
1778For a large project, you would probably have several source files, and use
1779@code{make} to arrange to recompile them.
1780
1781With all the source in a single file, you use the following command to
1782convert it into a parser file:
1783
1784@example
1785bison @var{file}.y
1786@end example
1787
1788@noindent
1789In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
1790@sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
1791removing the @samp{.y} from the original file name. The file output by
1792Bison contains the source code for @code{yyparse}. The additional
1793functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1794are copied verbatim to the output.
1795
1796@node Rpcalc Compile
1797@subsection Compiling the Parser File
1798@cindex compiling the parser
1799
1800Here is how to compile and run the parser file:
1801
1802@example
1803@group
1804# @r{List files in current directory.}
1805$ @kbd{ls}
1806rpcalc.tab.c rpcalc.y
1807@end group
1808
1809@group
1810# @r{Compile the Bison parser.}
1811# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
1812$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
1813@end group
1814
1815@group
1816# @r{List files again.}
1817$ @kbd{ls}
1818rpcalc rpcalc.tab.c rpcalc.y
1819@end group
1820@end example
1821
1822The file @file{rpcalc} now contains the executable code. Here is an
1823example session using @code{rpcalc}.
1824
1825@example
1826$ @kbd{rpcalc}
1827@kbd{4 9 +}
182813
1829@kbd{3 7 + 3 4 5 *+-}
1830-13
1831@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
183213
1833@kbd{5 6 / 4 n +}
1834-3.166666667
1835@kbd{3 4 ^} @r{Exponentiation}
183681
1837@kbd{^D} @r{End-of-file indicator}
1838$
1839@end example
1840
1841@node Infix Calc
1842@section Infix Notation Calculator: @code{calc}
1843@cindex infix notation calculator
1844@cindex @code{calc}
1845@cindex calculator, infix notation
1846
1847We now modify rpcalc to handle infix operators instead of postfix. Infix
1848notation involves the concept of operator precedence and the need for
1849parentheses nested to arbitrary depth. Here is the Bison code for
1850@file{calc.y}, an infix desk-top calculator.
1851
1852@example
1853/* Infix notation calculator. */
1854
1855%@{
1856 #define YYSTYPE double
1857 #include <math.h>
1858 #include <stdio.h>
1859 int yylex (void);
1860 void yyerror (char const *);
1861%@}
1862
1863/* Bison declarations. */
1864%token NUM
1865%left '-' '+'
1866%left '*' '/'
1867%precedence NEG /* negation--unary minus */
1868%right '^' /* exponentiation */
1869
1870%% /* The grammar follows. */
1871input: /* empty */
1872 | input line
1873;
1874
1875line: '\n'
1876 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1877;
1878
1879exp: NUM @{ $$ = $1; @}
1880 | exp '+' exp @{ $$ = $1 + $3; @}
1881 | exp '-' exp @{ $$ = $1 - $3; @}
1882 | exp '*' exp @{ $$ = $1 * $3; @}
1883 | exp '/' exp @{ $$ = $1 / $3; @}
1884 | '-' exp %prec NEG @{ $$ = -$2; @}
1885 | exp '^' exp @{ $$ = pow ($1, $3); @}
1886 | '(' exp ')' @{ $$ = $2; @}
1887;
1888%%
1889@end example
1890
1891@noindent
1892The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1893same as before.
1894
1895There are two important new features shown in this code.
1896
1897In the second section (Bison declarations), @code{%left} declares token
1898types and says they are left-associative operators. The declarations
1899@code{%left} and @code{%right} (right associativity) take the place of
1900@code{%token} which is used to declare a token type name without
1901associativity/precedence. (These tokens are single-character literals, which
1902ordinarily don't need to be declared. We declare them here to specify
1903the associativity/precedence.)
1904
1905Operator precedence is determined by the line ordering of the
1906declarations; the higher the line number of the declaration (lower on
1907the page or screen), the higher the precedence. Hence, exponentiation
1908has the highest precedence, unary minus (@code{NEG}) is next, followed
1909by @samp{*} and @samp{/}, and so on. Unary minus is not associative,
1910only precedence matters (@code{%precedence}. @xref{Precedence, ,Operator
1911Precedence}.
1912
1913The other important new feature is the @code{%prec} in the grammar
1914section for the unary minus operator. The @code{%prec} simply instructs
1915Bison that the rule @samp{| '-' exp} has the same precedence as
1916@code{NEG}---in this case the next-to-highest. @xref{Contextual
1917Precedence, ,Context-Dependent Precedence}.
1918
1919Here is a sample run of @file{calc.y}:
1920
1921@need 500
1922@example
1923$ @kbd{calc}
1924@kbd{4 + 4.5 - (34/(8*3+-3))}
19256.880952381
1926@kbd{-56 + 2}
1927-54
1928@kbd{3 ^ 2}
19299
1930@end example
1931
1932@node Simple Error Recovery
1933@section Simple Error Recovery
1934@cindex error recovery, simple
1935
1936Up to this point, this manual has not addressed the issue of @dfn{error
1937recovery}---how to continue parsing after the parser detects a syntax
1938error. All we have handled is error reporting with @code{yyerror}.
1939Recall that by default @code{yyparse} returns after calling
1940@code{yyerror}. This means that an erroneous input line causes the
1941calculator program to exit. Now we show how to rectify this deficiency.
1942
1943The Bison language itself includes the reserved word @code{error}, which
1944may be included in the grammar rules. In the example below it has
1945been added to one of the alternatives for @code{line}:
1946
1947@example
1948@group
1949line: '\n'
1950 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1951 | error '\n' @{ yyerrok; @}
1952;
1953@end group
1954@end example
1955
1956This addition to the grammar allows for simple error recovery in the
1957event of a syntax error. If an expression that cannot be evaluated is
1958read, the error will be recognized by the third rule for @code{line},
1959and parsing will continue. (The @code{yyerror} function is still called
1960upon to print its message as well.) The action executes the statement
1961@code{yyerrok}, a macro defined automatically by Bison; its meaning is
1962that error recovery is complete (@pxref{Error Recovery}). Note the
1963difference between @code{yyerrok} and @code{yyerror}; neither one is a
1964misprint.
1965
1966This form of error recovery deals with syntax errors. There are other
1967kinds of errors; for example, division by zero, which raises an exception
1968signal that is normally fatal. A real calculator program must handle this
1969signal and use @code{longjmp} to return to @code{main} and resume parsing
1970input lines; it would also have to discard the rest of the current line of
1971input. We won't discuss this issue further because it is not specific to
1972Bison programs.
1973
1974@node Location Tracking Calc
1975@section Location Tracking Calculator: @code{ltcalc}
1976@cindex location tracking calculator
1977@cindex @code{ltcalc}
1978@cindex calculator, location tracking
1979
1980This example extends the infix notation calculator with location
1981tracking. This feature will be used to improve the error messages. For
1982the sake of clarity, this example is a simple integer calculator, since
1983most of the work needed to use locations will be done in the lexical
1984analyzer.
1985
1986@menu
1987* Ltcalc Declarations:: Bison and C declarations for ltcalc.
1988* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
1989* Ltcalc Lexer:: The lexical analyzer.
1990@end menu
1991
1992@node Ltcalc Declarations
1993@subsection Declarations for @code{ltcalc}
1994
1995The C and Bison declarations for the location tracking calculator are
1996the same as the declarations for the infix notation calculator.
1997
1998@example
1999/* Location tracking calculator. */
2000
2001%@{
2002 #define YYSTYPE int
2003 #include <math.h>
2004 int yylex (void);
2005 void yyerror (char const *);
2006%@}
2007
2008/* Bison declarations. */
2009%token NUM
2010
2011%left '-' '+'
2012%left '*' '/'
2013%precedence NEG
2014%right '^'
2015
2016%% /* The grammar follows. */
2017@end example
2018
2019@noindent
2020Note there are no declarations specific to locations. Defining a data
2021type for storing locations is not needed: we will use the type provided
2022by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2023four member structure with the following integer fields:
2024@code{first_line}, @code{first_column}, @code{last_line} and
2025@code{last_column}. By conventions, and in accordance with the GNU
2026Coding Standards and common practice, the line and column count both
2027start at 1.
2028
2029@node Ltcalc Rules
2030@subsection Grammar Rules for @code{ltcalc}
2031
2032Whether handling locations or not has no effect on the syntax of your
2033language. Therefore, grammar rules for this example will be very close
2034to those of the previous example: we will only modify them to benefit
2035from the new information.
2036
2037Here, we will use locations to report divisions by zero, and locate the
2038wrong expressions or subexpressions.
2039
2040@example
2041@group
2042input : /* empty */
2043 | input line
2044;
2045@end group
2046
2047@group
2048line : '\n'
2049 | exp '\n' @{ printf ("%d\n", $1); @}
2050;
2051@end group
2052
2053@group
2054exp : NUM @{ $$ = $1; @}
2055 | exp '+' exp @{ $$ = $1 + $3; @}
2056 | exp '-' exp @{ $$ = $1 - $3; @}
2057 | exp '*' exp @{ $$ = $1 * $3; @}
2058@end group
2059@group
2060 | exp '/' exp
2061 @{
2062 if ($3)
2063 $$ = $1 / $3;
2064 else
2065 @{
2066 $$ = 1;
2067 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2068 @@3.first_line, @@3.first_column,
2069 @@3.last_line, @@3.last_column);
2070 @}
2071 @}
2072@end group
2073@group
2074 | '-' exp %prec NEG @{ $$ = -$2; @}
2075 | exp '^' exp @{ $$ = pow ($1, $3); @}
2076 | '(' exp ')' @{ $$ = $2; @}
2077@end group
2078@end example
2079
2080This code shows how to reach locations inside of semantic actions, by
2081using the pseudo-variables @code{@@@var{n}} for rule components, and the
2082pseudo-variable @code{@@$} for groupings.
2083
2084We don't need to assign a value to @code{@@$}: the output parser does it
2085automatically. By default, before executing the C code of each action,
2086@code{@@$} is set to range from the beginning of @code{@@1} to the end
2087of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2088can be redefined (@pxref{Location Default Action, , Default Action for
2089Locations}), and for very specific rules, @code{@@$} can be computed by
2090hand.
2091
2092@node Ltcalc Lexer
2093@subsection The @code{ltcalc} Lexical Analyzer.
2094
2095Until now, we relied on Bison's defaults to enable location
2096tracking. The next step is to rewrite the lexical analyzer, and make it
2097able to feed the parser with the token locations, as it already does for
2098semantic values.
2099
2100To this end, we must take into account every single character of the
2101input text, to avoid the computed locations of being fuzzy or wrong:
2102
2103@example
2104@group
2105int
2106yylex (void)
2107@{
2108 int c;
2109@end group
2110
2111@group
2112 /* Skip white space. */
2113 while ((c = getchar ()) == ' ' || c == '\t')
2114 ++yylloc.last_column;
2115@end group
2116
2117@group
2118 /* Step. */
2119 yylloc.first_line = yylloc.last_line;
2120 yylloc.first_column = yylloc.last_column;
2121@end group
2122
2123@group
2124 /* Process numbers. */
2125 if (isdigit (c))
2126 @{
2127 yylval = c - '0';
2128 ++yylloc.last_column;
2129 while (isdigit (c = getchar ()))
2130 @{
2131 ++yylloc.last_column;
2132 yylval = yylval * 10 + c - '0';
2133 @}
2134 ungetc (c, stdin);
2135 return NUM;
2136 @}
2137@end group
2138
2139 /* Return end-of-input. */
2140 if (c == EOF)
2141 return 0;
2142
2143 /* Return a single char, and update location. */
2144 if (c == '\n')
2145 @{
2146 ++yylloc.last_line;
2147 yylloc.last_column = 0;
2148 @}
2149 else
2150 ++yylloc.last_column;
2151 return c;
2152@}
2153@end example
2154
2155Basically, the lexical analyzer performs the same processing as before:
2156it skips blanks and tabs, and reads numbers or single-character tokens.
2157In addition, it updates @code{yylloc}, the global variable (of type
2158@code{YYLTYPE}) containing the token's location.
2159
2160Now, each time this function returns a token, the parser has its number
2161as well as its semantic value, and its location in the text. The last
2162needed change is to initialize @code{yylloc}, for example in the
2163controlling function:
2164
2165@example
2166@group
2167int
2168main (void)
2169@{
2170 yylloc.first_line = yylloc.last_line = 1;
2171 yylloc.first_column = yylloc.last_column = 0;
2172 return yyparse ();
2173@}
2174@end group
2175@end example
2176
2177Remember that computing locations is not a matter of syntax. Every
2178character must be associated to a location update, whether it is in
2179valid input, in comments, in literal strings, and so on.
2180
2181@node Multi-function Calc
2182@section Multi-Function Calculator: @code{mfcalc}
2183@cindex multi-function calculator
2184@cindex @code{mfcalc}
2185@cindex calculator, multi-function
2186
2187Now that the basics of Bison have been discussed, it is time to move on to
2188a more advanced problem. The above calculators provided only five
2189functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2190be nice to have a calculator that provides other mathematical functions such
2191as @code{sin}, @code{cos}, etc.
2192
2193It is easy to add new operators to the infix calculator as long as they are
2194only single-character literals. The lexical analyzer @code{yylex} passes
2195back all nonnumeric characters as tokens, so new grammar rules suffice for
2196adding a new operator. But we want something more flexible: built-in
2197functions whose syntax has this form:
2198
2199@example
2200@var{function_name} (@var{argument})
2201@end example
2202
2203@noindent
2204At the same time, we will add memory to the calculator, by allowing you
2205to create named variables, store values in them, and use them later.
2206Here is a sample session with the multi-function calculator:
2207
2208@example
2209$ @kbd{mfcalc}
2210@kbd{pi = 3.141592653589}
22113.1415926536
2212@kbd{sin(pi)}
22130.0000000000
2214@kbd{alpha = beta1 = 2.3}
22152.3000000000
2216@kbd{alpha}
22172.3000000000
2218@kbd{ln(alpha)}
22190.8329091229
2220@kbd{exp(ln(beta1))}
22212.3000000000
2222$
2223@end example
2224
2225Note that multiple assignment and nested function calls are permitted.
2226
2227@menu
2228* Mfcalc Declarations:: Bison declarations for multi-function calculator.
2229* Mfcalc Rules:: Grammar rules for the calculator.
2230* Mfcalc Symbol Table:: Symbol table management subroutines.
2231@end menu
2232
2233@node Mfcalc Declarations
2234@subsection Declarations for @code{mfcalc}
2235
2236Here are the C and Bison declarations for the multi-function calculator.
2237
2238@smallexample
2239@group
2240%@{
2241 #include <math.h> /* For math functions, cos(), sin(), etc. */
2242 #include "calc.h" /* Contains definition of `symrec'. */
2243 int yylex (void);
2244 void yyerror (char const *);
2245%@}
2246@end group
2247@group
2248%union @{
2249 double val; /* For returning numbers. */
2250 symrec *tptr; /* For returning symbol-table pointers. */
2251@}
2252@end group
2253%token <val> NUM /* Simple double precision number. */
2254%token <tptr> VAR FNCT /* Variable and Function. */
2255%type <val> exp
2256
2257@group
2258%right '='
2259%left '-' '+'
2260%left '*' '/'
2261%precedence NEG /* negation--unary minus */
2262%right '^' /* exponentiation */
2263@end group
2264%% /* The grammar follows. */
2265@end smallexample
2266
2267The above grammar introduces only two new features of the Bison language.
2268These features allow semantic values to have various data types
2269(@pxref{Multiple Types, ,More Than One Value Type}).
2270
2271The @code{%union} declaration specifies the entire list of possible types;
2272this is instead of defining @code{YYSTYPE}. The allowable types are now
2273double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2274the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2275
2276Since values can now have various types, it is necessary to associate a
2277type with each grammar symbol whose semantic value is used. These symbols
2278are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2279declarations are augmented with information about their data type (placed
2280between angle brackets).
2281
2282The Bison construct @code{%type} is used for declaring nonterminal
2283symbols, just as @code{%token} is used for declaring token types. We
2284have not used @code{%type} before because nonterminal symbols are
2285normally declared implicitly by the rules that define them. But
2286@code{exp} must be declared explicitly so we can specify its value type.
2287@xref{Type Decl, ,Nonterminal Symbols}.
2288
2289@node Mfcalc Rules
2290@subsection Grammar Rules for @code{mfcalc}
2291
2292Here are the grammar rules for the multi-function calculator.
2293Most of them are copied directly from @code{calc}; three rules,
2294those which mention @code{VAR} or @code{FNCT}, are new.
2295
2296@smallexample
2297@group
2298input: /* empty */
2299 | input line
2300;
2301@end group
2302
2303@group
2304line:
2305 '\n'
2306 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2307 | error '\n' @{ yyerrok; @}
2308;
2309@end group
2310
2311@group
2312exp: NUM @{ $$ = $1; @}
2313 | VAR @{ $$ = $1->value.var; @}
2314 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2315 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2316 | exp '+' exp @{ $$ = $1 + $3; @}
2317 | exp '-' exp @{ $$ = $1 - $3; @}
2318 | exp '*' exp @{ $$ = $1 * $3; @}
2319 | exp '/' exp @{ $$ = $1 / $3; @}
2320 | '-' exp %prec NEG @{ $$ = -$2; @}
2321 | exp '^' exp @{ $$ = pow ($1, $3); @}
2322 | '(' exp ')' @{ $$ = $2; @}
2323;
2324@end group
2325/* End of grammar. */
2326%%
2327@end smallexample
2328
2329@node Mfcalc Symbol Table
2330@subsection The @code{mfcalc} Symbol Table
2331@cindex symbol table example
2332
2333The multi-function calculator requires a symbol table to keep track of the
2334names and meanings of variables and functions. This doesn't affect the
2335grammar rules (except for the actions) or the Bison declarations, but it
2336requires some additional C functions for support.
2337
2338The symbol table itself consists of a linked list of records. Its
2339definition, which is kept in the header @file{calc.h}, is as follows. It
2340provides for either functions or variables to be placed in the table.
2341
2342@smallexample
2343@group
2344/* Function type. */
2345typedef double (*func_t) (double);
2346@end group
2347
2348@group
2349/* Data type for links in the chain of symbols. */
2350struct symrec
2351@{
2352 char *name; /* name of symbol */
2353 int type; /* type of symbol: either VAR or FNCT */
2354 union
2355 @{
2356 double var; /* value of a VAR */
2357 func_t fnctptr; /* value of a FNCT */
2358 @} value;
2359 struct symrec *next; /* link field */
2360@};
2361@end group
2362
2363@group
2364typedef struct symrec symrec;
2365
2366/* The symbol table: a chain of `struct symrec'. */
2367extern symrec *sym_table;
2368
2369symrec *putsym (char const *, int);
2370symrec *getsym (char const *);
2371@end group
2372@end smallexample
2373
2374The new version of @code{main} includes a call to @code{init_table}, a
2375function that initializes the symbol table. Here it is, and
2376@code{init_table} as well:
2377
2378@smallexample
2379#include <stdio.h>
2380
2381@group
2382/* Called by yyparse on error. */
2383void
2384yyerror (char const *s)
2385@{
2386 printf ("%s\n", s);
2387@}
2388@end group
2389
2390@group
2391struct init
2392@{
2393 char const *fname;
2394 double (*fnct) (double);
2395@};
2396@end group
2397
2398@group
2399struct init const arith_fncts[] =
2400@{
2401 "sin", sin,
2402 "cos", cos,
2403 "atan", atan,
2404 "ln", log,
2405 "exp", exp,
2406 "sqrt", sqrt,
2407 0, 0
2408@};
2409@end group
2410
2411@group
2412/* The symbol table: a chain of `struct symrec'. */
2413symrec *sym_table;
2414@end group
2415
2416@group
2417/* Put arithmetic functions in table. */
2418void
2419init_table (void)
2420@{
2421 int i;
2422 symrec *ptr;
2423 for (i = 0; arith_fncts[i].fname != 0; i++)
2424 @{
2425 ptr = putsym (arith_fncts[i].fname, FNCT);
2426 ptr->value.fnctptr = arith_fncts[i].fnct;
2427 @}
2428@}
2429@end group
2430
2431@group
2432int
2433main (void)
2434@{
2435 init_table ();
2436 return yyparse ();
2437@}
2438@end group
2439@end smallexample
2440
2441By simply editing the initialization list and adding the necessary include
2442files, you can add additional functions to the calculator.
2443
2444Two important functions allow look-up and installation of symbols in the
2445symbol table. The function @code{putsym} is passed a name and the type
2446(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2447linked to the front of the list, and a pointer to the object is returned.
2448The function @code{getsym} is passed the name of the symbol to look up. If
2449found, a pointer to that symbol is returned; otherwise zero is returned.
2450
2451@smallexample
2452symrec *
2453putsym (char const *sym_name, int sym_type)
2454@{
2455 symrec *ptr;
2456 ptr = (symrec *) malloc (sizeof (symrec));
2457 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2458 strcpy (ptr->name,sym_name);
2459 ptr->type = sym_type;
2460 ptr->value.var = 0; /* Set value to 0 even if fctn. */
2461 ptr->next = (struct symrec *)sym_table;
2462 sym_table = ptr;
2463 return ptr;
2464@}
2465
2466symrec *
2467getsym (char const *sym_name)
2468@{
2469 symrec *ptr;
2470 for (ptr = sym_table; ptr != (symrec *) 0;
2471 ptr = (symrec *)ptr->next)
2472 if (strcmp (ptr->name,sym_name) == 0)
2473 return ptr;
2474 return 0;
2475@}
2476@end smallexample
2477
2478The function @code{yylex} must now recognize variables, numeric values, and
2479the single-character arithmetic operators. Strings of alphanumeric
2480characters with a leading letter are recognized as either variables or
2481functions depending on what the symbol table says about them.
2482
2483The string is passed to @code{getsym} for look up in the symbol table. If
2484the name appears in the table, a pointer to its location and its type
2485(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2486already in the table, then it is installed as a @code{VAR} using
2487@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
2488returned to @code{yyparse}.
2489
2490No change is needed in the handling of numeric values and arithmetic
2491operators in @code{yylex}.
2492
2493@smallexample
2494@group
2495#include <ctype.h>
2496@end group
2497
2498@group
2499int
2500yylex (void)
2501@{
2502 int c;
2503
2504 /* Ignore white space, get first nonwhite character. */
2505 while ((c = getchar ()) == ' ' || c == '\t');
2506
2507 if (c == EOF)
2508 return 0;
2509@end group
2510
2511@group
2512 /* Char starts a number => parse the number. */
2513 if (c == '.' || isdigit (c))
2514 @{
2515 ungetc (c, stdin);
2516 scanf ("%lf", &yylval.val);
2517 return NUM;
2518 @}
2519@end group
2520
2521@group
2522 /* Char starts an identifier => read the name. */
2523 if (isalpha (c))
2524 @{
2525 symrec *s;
2526 static char *symbuf = 0;
2527 static int length = 0;
2528 int i;
2529@end group
2530
2531@group
2532 /* Initially make the buffer long enough
2533 for a 40-character symbol name. */
2534 if (length == 0)
2535 length = 40, symbuf = (char *)malloc (length + 1);
2536
2537 i = 0;
2538 do
2539@end group
2540@group
2541 @{
2542 /* If buffer is full, make it bigger. */
2543 if (i == length)
2544 @{
2545 length *= 2;
2546 symbuf = (char *) realloc (symbuf, length + 1);
2547 @}
2548 /* Add this character to the buffer. */
2549 symbuf[i++] = c;
2550 /* Get another character. */
2551 c = getchar ();
2552 @}
2553@end group
2554@group
2555 while (isalnum (c));
2556
2557 ungetc (c, stdin);
2558 symbuf[i] = '\0';
2559@end group
2560
2561@group
2562 s = getsym (symbuf);
2563 if (s == 0)
2564 s = putsym (symbuf, VAR);
2565 yylval.tptr = s;
2566 return s->type;
2567 @}
2568
2569 /* Any other character is a token by itself. */
2570 return c;
2571@}
2572@end group
2573@end smallexample
2574
2575This program is both powerful and flexible. You may easily add new
2576functions, and it is a simple job to modify this code to install
2577predefined variables such as @code{pi} or @code{e} as well.
2578
2579@node Exercises
2580@section Exercises
2581@cindex exercises
2582
2583@enumerate
2584@item
2585Add some new functions from @file{math.h} to the initialization list.
2586
2587@item
2588Add another array that contains constants and their values. Then
2589modify @code{init_table} to add these constants to the symbol table.
2590It will be easiest to give the constants type @code{VAR}.
2591
2592@item
2593Make the program report an error if the user refers to an
2594uninitialized variable in any way except to store a value in it.
2595@end enumerate
2596
2597@node Grammar File
2598@chapter Bison Grammar Files
2599
2600Bison takes as input a context-free grammar specification and produces a
2601C-language function that recognizes correct instances of the grammar.
2602
2603The Bison grammar input file conventionally has a name ending in @samp{.y}.
2604@xref{Invocation, ,Invoking Bison}.
2605
2606@menu
2607* Grammar Outline:: Overall layout of the grammar file.
2608* Symbols:: Terminal and nonterminal symbols.
2609* Rules:: How to write grammar rules.
2610* Recursion:: Writing recursive rules.
2611* Semantics:: Semantic values and actions.
2612* Locations:: Locations and actions.
2613* Declarations:: All kinds of Bison declarations are described here.
2614* Multiple Parsers:: Putting more than one Bison parser in one program.
2615@end menu
2616
2617@node Grammar Outline
2618@section Outline of a Bison Grammar
2619
2620A Bison grammar file has four main sections, shown here with the
2621appropriate delimiters:
2622
2623@example
2624%@{
2625 @var{Prologue}
2626%@}
2627
2628@var{Bison declarations}
2629
2630%%
2631@var{Grammar rules}
2632%%
2633
2634@var{Epilogue}
2635@end example
2636
2637Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2638As a @acronym{GNU} extension, @samp{//} introduces a comment that
2639continues until end of line.
2640
2641@menu
2642* Prologue:: Syntax and usage of the prologue.
2643* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
2644* Bison Declarations:: Syntax and usage of the Bison declarations section.
2645* Grammar Rules:: Syntax and usage of the grammar rules section.
2646* Epilogue:: Syntax and usage of the epilogue.
2647@end menu
2648
2649@node Prologue
2650@subsection The prologue
2651@cindex declarations section
2652@cindex Prologue
2653@cindex declarations
2654
2655The @var{Prologue} section contains macro definitions and declarations
2656of functions and variables that are used in the actions in the grammar
2657rules. These are copied to the beginning of the parser file so that
2658they precede the definition of @code{yyparse}. You can use
2659@samp{#include} to get the declarations from a header file. If you
2660don't need any C declarations, you may omit the @samp{%@{} and
2661@samp{%@}} delimiters that bracket this section.
2662
2663The @var{Prologue} section is terminated by the first occurrence
2664of @samp{%@}} that is outside a comment, a string literal, or a
2665character constant.
2666
2667You may have more than one @var{Prologue} section, intermixed with the
2668@var{Bison declarations}. This allows you to have C and Bison
2669declarations that refer to each other. For example, the @code{%union}
2670declaration may use types defined in a header file, and you may wish to
2671prototype functions that take arguments of type @code{YYSTYPE}. This
2672can be done with two @var{Prologue} blocks, one before and one after the
2673@code{%union} declaration.
2674
2675@smallexample
2676%@{
2677 #define _GNU_SOURCE
2678 #include <stdio.h>
2679 #include "ptypes.h"
2680%@}
2681
2682%union @{
2683 long int n;
2684 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2685@}
2686
2687%@{
2688 static void print_token_value (FILE *, int, YYSTYPE);
2689 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2690%@}
2691
2692@dots{}
2693@end smallexample
2694
2695When in doubt, it is usually safer to put prologue code before all
2696Bison declarations, rather than after. For example, any definitions
2697of feature test macros like @code{_GNU_SOURCE} or
2698@code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2699feature test macros can affect the behavior of Bison-generated
2700@code{#include} directives.
2701
2702@node Prologue Alternatives
2703@subsection Prologue Alternatives
2704@cindex Prologue Alternatives
2705
2706@findex %code
2707@findex %code requires
2708@findex %code provides
2709@findex %code top
2710
2711The functionality of @var{Prologue} sections can often be subtle and
2712inflexible.
2713As an alternative, Bison provides a %code directive with an explicit qualifier
2714field, which identifies the purpose of the code and thus the location(s) where
2715Bison should generate it.
2716For C/C++, the qualifier can be omitted for the default location, or it can be
2717one of @code{requires}, @code{provides}, @code{top}.
2718@xref{Decl Summary,,%code}.
2719
2720Look again at the example of the previous section:
2721
2722@smallexample
2723%@{
2724 #define _GNU_SOURCE
2725 #include <stdio.h>
2726 #include "ptypes.h"
2727%@}
2728
2729%union @{
2730 long int n;
2731 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2732@}
2733
2734%@{
2735 static void print_token_value (FILE *, int, YYSTYPE);
2736 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2737%@}
2738
2739@dots{}
2740@end smallexample
2741
2742@noindent
2743Notice that there are two @var{Prologue} sections here, but there's a subtle
2744distinction between their functionality.
2745For example, if you decide to override Bison's default definition for
2746@code{YYLTYPE}, in which @var{Prologue} section should you write your new
2747definition?
2748You should write it in the first since Bison will insert that code into the
2749parser source code file @emph{before} the default @code{YYLTYPE} definition.
2750In which @var{Prologue} section should you prototype an internal function,
2751@code{trace_token}, that accepts @code{YYLTYPE} and @code{yytokentype} as
2752arguments?
2753You should prototype it in the second since Bison will insert that code
2754@emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2755
2756This distinction in functionality between the two @var{Prologue} sections is
2757established by the appearance of the @code{%union} between them.
2758This behavior raises a few questions.
2759First, why should the position of a @code{%union} affect definitions related to
2760@code{YYLTYPE} and @code{yytokentype}?
2761Second, what if there is no @code{%union}?
2762In that case, the second kind of @var{Prologue} section is not available.
2763This behavior is not intuitive.
2764
2765To avoid this subtle @code{%union} dependency, rewrite the example using a
2766@code{%code top} and an unqualified @code{%code}.
2767Let's go ahead and add the new @code{YYLTYPE} definition and the
2768@code{trace_token} prototype at the same time:
2769
2770@smallexample
2771%code top @{
2772 #define _GNU_SOURCE
2773 #include <stdio.h>
2774
2775 /* WARNING: The following code really belongs
2776 * in a `%code requires'; see below. */
2777
2778 #include "ptypes.h"
2779 #define YYLTYPE YYLTYPE
2780 typedef struct YYLTYPE
2781 @{
2782 int first_line;
2783 int first_column;
2784 int last_line;
2785 int last_column;
2786 char *filename;
2787 @} YYLTYPE;
2788@}
2789
2790%union @{
2791 long int n;
2792 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2793@}
2794
2795%code @{
2796 static void print_token_value (FILE *, int, YYSTYPE);
2797 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2798 static void trace_token (enum yytokentype token, YYLTYPE loc);
2799@}
2800
2801@dots{}
2802@end smallexample
2803
2804@noindent
2805In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2806functionality as the two kinds of @var{Prologue} sections, but it's always
2807explicit which kind you intend.
2808Moreover, both kinds are always available even in the absence of @code{%union}.
2809
2810The @code{%code top} block above logically contains two parts.
2811The first two lines before the warning need to appear near the top of the
2812parser source code file.
2813The first line after the warning is required by @code{YYSTYPE} and thus also
2814needs to appear in the parser source code file.
2815However, if you've instructed Bison to generate a parser header file
2816(@pxref{Decl Summary, ,%defines}), you probably want that line to appear before
2817the @code{YYSTYPE} definition in that header file as well.
2818The @code{YYLTYPE} definition should also appear in the parser header file to
2819override the default @code{YYLTYPE} definition there.
2820
2821In other words, in the @code{%code top} block above, all but the first two
2822lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2823definitions.
2824Thus, they belong in one or more @code{%code requires}:
2825
2826@smallexample
2827%code top @{
2828 #define _GNU_SOURCE
2829 #include <stdio.h>
2830@}
2831
2832%code requires @{
2833 #include "ptypes.h"
2834@}
2835%union @{
2836 long int n;
2837 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2838@}
2839
2840%code requires @{
2841 #define YYLTYPE YYLTYPE
2842 typedef struct YYLTYPE
2843 @{
2844 int first_line;
2845 int first_column;
2846 int last_line;
2847 int last_column;
2848 char *filename;
2849 @} YYLTYPE;
2850@}
2851
2852%code @{
2853 static void print_token_value (FILE *, int, YYSTYPE);
2854 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2855 static void trace_token (enum yytokentype token, YYLTYPE loc);
2856@}
2857
2858@dots{}
2859@end smallexample
2860
2861@noindent
2862Now Bison will insert @code{#include "ptypes.h"} and the new @code{YYLTYPE}
2863definition before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
2864definitions in both the parser source code file and the parser header file.
2865(By the same reasoning, @code{%code requires} would also be the appropriate
2866place to write your own definition for @code{YYSTYPE}.)
2867
2868When you are writing dependency code for @code{YYSTYPE} and @code{YYLTYPE}, you
2869should prefer @code{%code requires} over @code{%code top} regardless of whether
2870you instruct Bison to generate a parser header file.
2871When you are writing code that you need Bison to insert only into the parser
2872source code file and that has no special need to appear at the top of that
2873file, you should prefer the unqualified @code{%code} over @code{%code top}.
2874These practices will make the purpose of each block of your code explicit to
2875Bison and to other developers reading your grammar file.
2876Following these practices, we expect the unqualified @code{%code} and
2877@code{%code requires} to be the most important of the four @var{Prologue}
2878alternatives.
2879
2880At some point while developing your parser, you might decide to provide
2881@code{trace_token} to modules that are external to your parser.
2882Thus, you might wish for Bison to insert the prototype into both the parser
2883header file and the parser source code file.
2884Since this function is not a dependency required by @code{YYSTYPE} or
2885@code{YYLTYPE}, it doesn't make sense to move its prototype to a
2886@code{%code requires}.
2887More importantly, since it depends upon @code{YYLTYPE} and @code{yytokentype},
2888@code{%code requires} is not sufficient.
2889Instead, move its prototype from the unqualified @code{%code} to a
2890@code{%code provides}:
2891
2892@smallexample
2893%code top @{
2894 #define _GNU_SOURCE
2895 #include <stdio.h>
2896@}
2897
2898%code requires @{
2899 #include "ptypes.h"
2900@}
2901%union @{
2902 long int n;
2903 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2904@}
2905
2906%code requires @{
2907 #define YYLTYPE YYLTYPE
2908 typedef struct YYLTYPE
2909 @{
2910 int first_line;
2911 int first_column;
2912 int last_line;
2913 int last_column;
2914 char *filename;
2915 @} YYLTYPE;
2916@}
2917
2918%code provides @{
2919 void trace_token (enum yytokentype token, YYLTYPE loc);
2920@}
2921
2922%code @{
2923 static void print_token_value (FILE *, int, YYSTYPE);
2924 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2925@}
2926
2927@dots{}
2928@end smallexample
2929
2930@noindent
2931Bison will insert the @code{trace_token} prototype into both the parser header
2932file and the parser source code file after the definitions for
2933@code{yytokentype}, @code{YYLTYPE}, and @code{YYSTYPE}.
2934
2935The above examples are careful to write directives in an order that reflects
2936the layout of the generated parser source code and header files:
2937@code{%code top}, @code{%code requires}, @code{%code provides}, and then
2938@code{%code}.
2939While your grammar files may generally be easier to read if you also follow
2940this order, Bison does not require it.
2941Instead, Bison lets you choose an organization that makes sense to you.
2942
2943You may declare any of these directives multiple times in the grammar file.
2944In that case, Bison concatenates the contained code in declaration order.
2945This is the only way in which the position of one of these directives within
2946the grammar file affects its functionality.
2947
2948The result of the previous two properties is greater flexibility in how you may
2949organize your grammar file.
2950For example, you may organize semantic-type-related directives by semantic
2951type:
2952
2953@smallexample
2954%code requires @{ #include "type1.h" @}
2955%union @{ type1 field1; @}
2956%destructor @{ type1_free ($$); @} <field1>
2957%printer @{ type1_print ($$); @} <field1>
2958
2959%code requires @{ #include "type2.h" @}
2960%union @{ type2 field2; @}
2961%destructor @{ type2_free ($$); @} <field2>
2962%printer @{ type2_print ($$); @} <field2>
2963@end smallexample
2964
2965@noindent
2966You could even place each of the above directive groups in the rules section of
2967the grammar file next to the set of rules that uses the associated semantic
2968type.
2969(In the rules section, you must terminate each of those directives with a
2970semicolon.)
2971And you don't have to worry that some directive (like a @code{%union}) in the
2972definitions section is going to adversely affect their functionality in some
2973counter-intuitive manner just because it comes first.
2974Such an organization is not possible using @var{Prologue} sections.
2975
2976This section has been concerned with explaining the advantages of the four
2977@var{Prologue} alternatives over the original Yacc @var{Prologue}.
2978However, in most cases when using these directives, you shouldn't need to
2979think about all the low-level ordering issues discussed here.
2980Instead, you should simply use these directives to label each block of your
2981code according to its purpose and let Bison handle the ordering.
2982@code{%code} is the most generic label.
2983Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
2984as needed.
2985
2986@node Bison Declarations
2987@subsection The Bison Declarations Section
2988@cindex Bison declarations (introduction)
2989@cindex declarations, Bison (introduction)
2990
2991The @var{Bison declarations} section contains declarations that define
2992terminal and nonterminal symbols, specify precedence, and so on.
2993In some simple grammars you may not need any declarations.
2994@xref{Declarations, ,Bison Declarations}.
2995
2996@node Grammar Rules
2997@subsection The Grammar Rules Section
2998@cindex grammar rules section
2999@cindex rules section for grammar
3000
3001The @dfn{grammar rules} section contains one or more Bison grammar
3002rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
3003
3004There must always be at least one grammar rule, and the first
3005@samp{%%} (which precedes the grammar rules) may never be omitted even
3006if it is the first thing in the file.
3007
3008@node Epilogue
3009@subsection The epilogue
3010@cindex additional C code section
3011@cindex epilogue
3012@cindex C code, section for additional
3013
3014The @var{Epilogue} is copied verbatim to the end of the parser file, just as
3015the @var{Prologue} is copied to the beginning. This is the most convenient
3016place to put anything that you want to have in the parser file but which need
3017not come before the definition of @code{yyparse}. For example, the
3018definitions of @code{yylex} and @code{yyerror} often go here. Because
3019C requires functions to be declared before being used, you often need
3020to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
3021even if you define them in the Epilogue.
3022@xref{Interface, ,Parser C-Language Interface}.
3023
3024If the last section is empty, you may omit the @samp{%%} that separates it
3025from the grammar rules.
3026
3027The Bison parser itself contains many macros and identifiers whose names
3028start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3029any such names (except those documented in this manual) in the epilogue
3030of the grammar file.
3031
3032@node Symbols
3033@section Symbols, Terminal and Nonterminal
3034@cindex nonterminal symbol
3035@cindex terminal symbol
3036@cindex token type
3037@cindex symbol
3038
3039@dfn{Symbols} in Bison grammars represent the grammatical classifications
3040of the language.
3041
3042A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3043class of syntactically equivalent tokens. You use the symbol in grammar
3044rules to mean that a token in that class is allowed. The symbol is
3045represented in the Bison parser by a numeric code, and the @code{yylex}
3046function returns a token type code to indicate what kind of token has
3047been read. You don't need to know what the code value is; you can use
3048the symbol to stand for it.
3049
3050A @dfn{nonterminal symbol} stands for a class of syntactically
3051equivalent groupings. The symbol name is used in writing grammar rules.
3052By convention, it should be all lower case.
3053
3054Symbol names can contain letters, underscores, periods, dashes, and (not
3055at the beginning) digits. Dashes in symbol names are a GNU
3056extension, incompatible with @acronym{POSIX} Yacc. Terminal symbols
3057that contain periods or dashes make little sense: since they are not
3058valid symbols (in most programming languages) they are not exported as
3059token names.
3060
3061There are three ways of writing terminal symbols in the grammar:
3062
3063@itemize @bullet
3064@item
3065A @dfn{named token type} is written with an identifier, like an
3066identifier in C@. By convention, it should be all upper case. Each
3067such name must be defined with a Bison declaration such as
3068@code{%token}. @xref{Token Decl, ,Token Type Names}.
3069
3070@item
3071@cindex character token
3072@cindex literal token
3073@cindex single-character literal
3074A @dfn{character token type} (or @dfn{literal character token}) is
3075written in the grammar using the same syntax used in C for character
3076constants; for example, @code{'+'} is a character token type. A
3077character token type doesn't need to be declared unless you need to
3078specify its semantic value data type (@pxref{Value Type, ,Data Types of
3079Semantic Values}), associativity, or precedence (@pxref{Precedence,
3080,Operator Precedence}).
3081
3082By convention, a character token type is used only to represent a
3083token that consists of that particular character. Thus, the token
3084type @code{'+'} is used to represent the character @samp{+} as a
3085token. Nothing enforces this convention, but if you depart from it,
3086your program will confuse other readers.
3087
3088All the usual escape sequences used in character literals in C can be
3089used in Bison as well, but you must not use the null character as a
3090character literal because its numeric code, zero, signifies
3091end-of-input (@pxref{Calling Convention, ,Calling Convention
3092for @code{yylex}}). Also, unlike standard C, trigraphs have no
3093special meaning in Bison character literals, nor is backslash-newline
3094allowed.
3095
3096@item
3097@cindex string token
3098@cindex literal string token
3099@cindex multicharacter literal
3100A @dfn{literal string token} is written like a C string constant; for
3101example, @code{"<="} is a literal string token. A literal string token
3102doesn't need to be declared unless you need to specify its semantic
3103value data type (@pxref{Value Type}), associativity, or precedence
3104(@pxref{Precedence}).
3105
3106You can associate the literal string token with a symbolic name as an
3107alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3108Declarations}). If you don't do that, the lexical analyzer has to
3109retrieve the token number for the literal string token from the
3110@code{yytname} table (@pxref{Calling Convention}).
3111
3112@strong{Warning}: literal string tokens do not work in Yacc.
3113
3114By convention, a literal string token is used only to represent a token
3115that consists of that particular string. Thus, you should use the token
3116type @code{"<="} to represent the string @samp{<=} as a token. Bison
3117does not enforce this convention, but if you depart from it, people who
3118read your program will be confused.
3119
3120All the escape sequences used in string literals in C can be used in
3121Bison as well, except that you must not use a null character within a
3122string literal. Also, unlike Standard C, trigraphs have no special
3123meaning in Bison string literals, nor is backslash-newline allowed. A
3124literal string token must contain two or more characters; for a token
3125containing just one character, use a character token (see above).
3126@end itemize
3127
3128How you choose to write a terminal symbol has no effect on its
3129grammatical meaning. That depends only on where it appears in rules and
3130on when the parser function returns that symbol.
3131
3132The value returned by @code{yylex} is always one of the terminal
3133symbols, except that a zero or negative value signifies end-of-input.
3134Whichever way you write the token type in the grammar rules, you write
3135it the same way in the definition of @code{yylex}. The numeric code
3136for a character token type is simply the positive numeric code of the
3137character, so @code{yylex} can use the identical value to generate the
3138requisite code, though you may need to convert it to @code{unsigned
3139char} to avoid sign-extension on hosts where @code{char} is signed.
3140Each named token type becomes a C macro in
3141the parser file, so @code{yylex} can use the name to stand for the code.
3142(This is why periods don't make sense in terminal symbols.)
3143@xref{Calling Convention, ,Calling Convention for @code{yylex}}.
3144
3145If @code{yylex} is defined in a separate file, you need to arrange for the
3146token-type macro definitions to be available there. Use the @samp{-d}
3147option when you run Bison, so that it will write these macro definitions
3148into a separate header file @file{@var{name}.tab.h} which you can include
3149in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3150
3151If you want to write a grammar that is portable to any Standard C
3152host, you must use only nonnull character tokens taken from the basic
3153execution character set of Standard C@. This set consists of the ten
3154digits, the 52 lower- and upper-case English letters, and the
3155characters in the following C-language string:
3156
3157@example
3158"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3159@end example
3160
3161The @code{yylex} function and Bison must use a consistent character set
3162and encoding for character tokens. For example, if you run Bison in an
3163@acronym{ASCII} environment, but then compile and run the resulting
3164program in an environment that uses an incompatible character set like
3165@acronym{EBCDIC}, the resulting program may not work because the tables
3166generated by Bison will assume @acronym{ASCII} numeric values for
3167character tokens. It is standard practice for software distributions to
3168contain C source files that were generated by Bison in an
3169@acronym{ASCII} environment, so installers on platforms that are
3170incompatible with @acronym{ASCII} must rebuild those files before
3171compiling them.
3172
3173The symbol @code{error} is a terminal symbol reserved for error recovery
3174(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
3175In particular, @code{yylex} should never return this value. The default
3176value of the error token is 256, unless you explicitly assigned 256 to
3177one of your tokens with a @code{%token} declaration.
3178
3179@node Rules
3180@section Syntax of Grammar Rules
3181@cindex rule syntax
3182@cindex grammar rule syntax
3183@cindex syntax of grammar rules
3184
3185A Bison grammar rule has the following general form:
3186
3187@example
3188@group
3189@var{result}: @var{components}@dots{}
3190 ;
3191@end group
3192@end example
3193
3194@noindent
3195where @var{result} is the nonterminal symbol that this rule describes,
3196and @var{components} are various terminal and nonterminal symbols that
3197are put together by this rule (@pxref{Symbols}).
3198
3199For example,
3200
3201@example
3202@group
3203exp: exp '+' exp
3204 ;
3205@end group
3206@end example
3207
3208@noindent
3209says that two groupings of type @code{exp}, with a @samp{+} token in between,
3210can be combined into a larger grouping of type @code{exp}.
3211
3212White space in rules is significant only to separate symbols. You can add
3213extra white space as you wish.
3214
3215Scattered among the components can be @var{actions} that determine
3216the semantics of the rule. An action looks like this:
3217
3218@example
3219@{@var{C statements}@}
3220@end example
3221
3222@noindent
3223@cindex braced code
3224This is an example of @dfn{braced code}, that is, C code surrounded by
3225braces, much like a compound statement in C@. Braced code can contain
3226any sequence of C tokens, so long as its braces are balanced. Bison
3227does not check the braced code for correctness directly; it merely
3228copies the code to the output file, where the C compiler can check it.
3229
3230Within braced code, the balanced-brace count is not affected by braces
3231within comments, string literals, or character constants, but it is
3232affected by the C digraphs @samp{<%} and @samp{%>} that represent
3233braces. At the top level braced code must be terminated by @samp{@}}
3234and not by a digraph. Bison does not look for trigraphs, so if braced
3235code uses trigraphs you should ensure that they do not affect the
3236nesting of braces or the boundaries of comments, string literals, or
3237character constants.
3238
3239Usually there is only one action and it follows the components.
3240@xref{Actions}.
3241
3242@findex |
3243Multiple rules for the same @var{result} can be written separately or can
3244be joined with the vertical-bar character @samp{|} as follows:
3245
3246@example
3247@group
3248@var{result}: @var{rule1-components}@dots{}
3249 | @var{rule2-components}@dots{}
3250 @dots{}
3251 ;
3252@end group
3253@end example
3254
3255@noindent
3256They are still considered distinct rules even when joined in this way.
3257
3258If @var{components} in a rule is empty, it means that @var{result} can
3259match the empty string. For example, here is how to define a
3260comma-separated sequence of zero or more @code{exp} groupings:
3261
3262@example
3263@group
3264expseq: /* empty */
3265 | expseq1
3266 ;
3267@end group
3268
3269@group
3270expseq1: exp
3271 | expseq1 ',' exp
3272 ;
3273@end group
3274@end example
3275
3276@noindent
3277It is customary to write a comment @samp{/* empty */} in each rule
3278with no components.
3279
3280@node Recursion
3281@section Recursive Rules
3282@cindex recursive rule
3283
3284A rule is called @dfn{recursive} when its @var{result} nonterminal
3285appears also on its right hand side. Nearly all Bison grammars need to
3286use recursion, because that is the only way to define a sequence of any
3287number of a particular thing. Consider this recursive definition of a
3288comma-separated sequence of one or more expressions:
3289
3290@example
3291@group
3292expseq1: exp
3293 | expseq1 ',' exp
3294 ;
3295@end group
3296@end example
3297
3298@cindex left recursion
3299@cindex right recursion
3300@noindent
3301Since the recursive use of @code{expseq1} is the leftmost symbol in the
3302right hand side, we call this @dfn{left recursion}. By contrast, here
3303the same construct is defined using @dfn{right recursion}:
3304
3305@example
3306@group
3307expseq1: exp
3308 | exp ',' expseq1
3309 ;
3310@end group
3311@end example
3312
3313@noindent
3314Any kind of sequence can be defined using either left recursion or right
3315recursion, but you should always use left recursion, because it can
3316parse a sequence of any number of elements with bounded stack space.
3317Right recursion uses up space on the Bison stack in proportion to the
3318number of elements in the sequence, because all the elements must be
3319shifted onto the stack before the rule can be applied even once.
3320@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3321of this.
3322
3323@cindex mutual recursion
3324@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3325rule does not appear directly on its right hand side, but does appear
3326in rules for other nonterminals which do appear on its right hand
3327side.
3328
3329For example:
3330
3331@example
3332@group
3333expr: primary
3334 | primary '+' primary
3335 ;
3336@end group
3337
3338@group
3339primary: constant
3340 | '(' expr ')'
3341 ;
3342@end group
3343@end example
3344
3345@noindent
3346defines two mutually-recursive nonterminals, since each refers to the
3347other.
3348
3349@node Semantics
3350@section Defining Language Semantics
3351@cindex defining language semantics
3352@cindex language semantics, defining
3353
3354The grammar rules for a language determine only the syntax. The semantics
3355are determined by the semantic values associated with various tokens and
3356groupings, and by the actions taken when various groupings are recognized.
3357
3358For example, the calculator calculates properly because the value
3359associated with each expression is the proper number; it adds properly
3360because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3361the numbers associated with @var{x} and @var{y}.
3362
3363@menu
3364* Value Type:: Specifying one data type for all semantic values.
3365* Multiple Types:: Specifying several alternative data types.
3366* Actions:: An action is the semantic definition of a grammar rule.
3367* Action Types:: Specifying data types for actions to operate on.
3368* Mid-Rule Actions:: Most actions go at the end of a rule.
3369 This says when, why and how to use the exceptional
3370 action in the middle of a rule.
3371* Named References:: Using named references in actions.
3372@end menu
3373
3374@node Value Type
3375@subsection Data Types of Semantic Values
3376@cindex semantic value type
3377@cindex value type, semantic
3378@cindex data types of semantic values
3379@cindex default data type
3380
3381In a simple program it may be sufficient to use the same data type for
3382the semantic values of all language constructs. This was true in the
3383@acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
3384Notation Calculator}).
3385
3386Bison normally uses the type @code{int} for semantic values if your
3387program uses the same data type for all language constructs. To
3388specify some other type, define @code{YYSTYPE} as a macro, like this:
3389
3390@example
3391#define YYSTYPE double
3392@end example
3393
3394@noindent
3395@code{YYSTYPE}'s replacement list should be a type name
3396that does not contain parentheses or square brackets.
3397This macro definition must go in the prologue of the grammar file
3398(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
3399
3400@node Multiple Types
3401@subsection More Than One Value Type
3402
3403In most programs, you will need different data types for different kinds
3404of tokens and groupings. For example, a numeric constant may need type
3405@code{int} or @code{long int}, while a string constant needs type
3406@code{char *}, and an identifier might need a pointer to an entry in the
3407symbol table.
3408
3409To use more than one data type for semantic values in one parser, Bison
3410requires you to do two things:
3411
3412@itemize @bullet
3413@item
3414Specify the entire collection of possible data types, either by using the
3415@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3416Value Types}), or by using a @code{typedef} or a @code{#define} to
3417define @code{YYSTYPE} to be a union type whose member names are
3418the type tags.
3419
3420@item
3421Choose one of those types for each symbol (terminal or nonterminal) for
3422which semantic values are used. This is done for tokens with the
3423@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3424and for groupings with the @code{%type} Bison declaration (@pxref{Type
3425Decl, ,Nonterminal Symbols}).
3426@end itemize
3427
3428@node Actions
3429@subsection Actions
3430@cindex action
3431@vindex $$
3432@vindex $@var{n}
3433@vindex $@var{name}
3434@vindex $[@var{name}]
3435
3436An action accompanies a syntactic rule and contains C code to be executed
3437each time an instance of that rule is recognized. The task of most actions
3438is to compute a semantic value for the grouping built by the rule from the
3439semantic values associated with tokens or smaller groupings.
3440
3441An action consists of braced code containing C statements, and can be
3442placed at any position in the rule;
3443it is executed at that position. Most rules have just one action at the
3444end of the rule, following all the components. Actions in the middle of
3445a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3446Actions, ,Actions in Mid-Rule}).
3447
3448The C code in an action can refer to the semantic values of the components
3449matched by the rule with the construct @code{$@var{n}}, which stands for
3450the value of the @var{n}th component. The semantic value for the grouping
3451being constructed is @code{$$}. In addition, the semantic values of
3452symbols can be accessed with the named references construct
3453@code{$@var{name}} or @code{$[@var{name}]}. Bison translates both of these
3454constructs into expressions of the appropriate type when it copies the
3455actions into the parser file. @code{$$} (or @code{$@var{name}}, when it
3456stands for the current grouping) is translated to a modifiable
3457lvalue, so it can be assigned to.
3458
3459Here is a typical example:
3460
3461@example
3462@group
3463exp: @dots{}
3464 | exp '+' exp
3465 @{ $$ = $1 + $3; @}
3466@end group
3467@end example
3468
3469Or, in terms of named references:
3470
3471@example
3472@group
3473exp[result]: @dots{}
3474 | exp[left] '+' exp[right]
3475 @{ $result = $left + $right; @}
3476@end group
3477@end example
3478
3479@noindent
3480This rule constructs an @code{exp} from two smaller @code{exp} groupings
3481connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3482(@code{$left} and @code{$right})
3483refer to the semantic values of the two component @code{exp} groupings,
3484which are the first and third symbols on the right hand side of the rule.
3485The sum is stored into @code{$$} (@code{$result}) so that it becomes the
3486semantic value of
3487the addition-expression just recognized by the rule. If there were a
3488useful semantic value associated with the @samp{+} token, it could be
3489referred to as @code{$2}.
3490
3491@xref{Named References,,Using Named References}, for more information
3492about using the named references construct.
3493
3494Note that the vertical-bar character @samp{|} is really a rule
3495separator, and actions are attached to a single rule. This is a
3496difference with tools like Flex, for which @samp{|} stands for either
3497``or'', or ``the same action as that of the next rule''. In the
3498following example, the action is triggered only when @samp{b} is found:
3499
3500@example
3501@group
3502a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3503@end group
3504@end example
3505
3506@cindex default action
3507If you don't specify an action for a rule, Bison supplies a default:
3508@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3509becomes the value of the whole rule. Of course, the default action is
3510valid only if the two data types match. There is no meaningful default
3511action for an empty rule; every empty rule must have an explicit action
3512unless the rule's value does not matter.
3513
3514@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3515to tokens and groupings on the stack @emph{before} those that match the
3516current rule. This is a very risky practice, and to use it reliably
3517you must be certain of the context in which the rule is applied. Here
3518is a case in which you can use this reliably:
3519
3520@example
3521@group
3522foo: expr bar '+' expr @{ @dots{} @}
3523 | expr bar '-' expr @{ @dots{} @}
3524 ;
3525@end group
3526
3527@group
3528bar: /* empty */
3529 @{ previous_expr = $0; @}
3530 ;
3531@end group
3532@end example
3533
3534As long as @code{bar} is used only in the fashion shown here, @code{$0}
3535always refers to the @code{expr} which precedes @code{bar} in the
3536definition of @code{foo}.
3537
3538@vindex yylval
3539It is also possible to access the semantic value of the lookahead token, if
3540any, from a semantic action.
3541This semantic value is stored in @code{yylval}.
3542@xref{Action Features, ,Special Features for Use in Actions}.
3543
3544@node Action Types
3545@subsection Data Types of Values in Actions
3546@cindex action data types
3547@cindex data types in actions
3548
3549If you have chosen a single data type for semantic values, the @code{$$}
3550and @code{$@var{n}} constructs always have that data type.
3551
3552If you have used @code{%union} to specify a variety of data types, then you
3553must declare a choice among these types for each terminal or nonterminal
3554symbol that can have a semantic value. Then each time you use @code{$$} or
3555@code{$@var{n}}, its data type is determined by which symbol it refers to
3556in the rule. In this example,
3557
3558@example
3559@group
3560exp: @dots{}
3561 | exp '+' exp
3562 @{ $$ = $1 + $3; @}
3563@end group
3564@end example
3565
3566@noindent
3567@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3568have the data type declared for the nonterminal symbol @code{exp}. If
3569@code{$2} were used, it would have the data type declared for the
3570terminal symbol @code{'+'}, whatever that might be.
3571
3572Alternatively, you can specify the data type when you refer to the value,
3573by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3574reference. For example, if you have defined types as shown here:
3575
3576@example
3577@group
3578%union @{
3579 int itype;
3580 double dtype;
3581@}
3582@end group
3583@end example
3584
3585@noindent
3586then you can write @code{$<itype>1} to refer to the first subunit of the
3587rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3588
3589@node Mid-Rule Actions
3590@subsection Actions in Mid-Rule
3591@cindex actions in mid-rule
3592@cindex mid-rule actions
3593
3594Occasionally it is useful to put an action in the middle of a rule.
3595These actions are written just like usual end-of-rule actions, but they
3596are executed before the parser even recognizes the following components.
3597
3598A mid-rule action may refer to the components preceding it using
3599@code{$@var{n}}, but it may not refer to subsequent components because
3600it is run before they are parsed.
3601
3602The mid-rule action itself counts as one of the components of the rule.
3603This makes a difference when there is another action later in the same rule
3604(and usually there is another at the end): you have to count the actions
3605along with the symbols when working out which number @var{n} to use in
3606@code{$@var{n}}.
3607
3608The mid-rule action can also have a semantic value. The action can set
3609its value with an assignment to @code{$$}, and actions later in the rule
3610can refer to the value using @code{$@var{n}}. Since there is no symbol
3611to name the action, there is no way to declare a data type for the value
3612in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3613specify a data type each time you refer to this value.
3614
3615There is no way to set the value of the entire rule with a mid-rule
3616action, because assignments to @code{$$} do not have that effect. The
3617only way to set the value for the entire rule is with an ordinary action
3618at the end of the rule.
3619
3620Here is an example from a hypothetical compiler, handling a @code{let}
3621statement that looks like @samp{let (@var{variable}) @var{statement}} and
3622serves to create a variable named @var{variable} temporarily for the
3623duration of @var{statement}. To parse this construct, we must put
3624@var{variable} into the symbol table while @var{statement} is parsed, then
3625remove it afterward. Here is how it is done:
3626
3627@example
3628@group
3629stmt: LET '(' var ')'
3630 @{ $<context>$ = push_context ();
3631 declare_variable ($3); @}
3632 stmt @{ $$ = $6;
3633 pop_context ($<context>5); @}
3634@end group
3635@end example
3636
3637@noindent
3638As soon as @samp{let (@var{variable})} has been recognized, the first
3639action is run. It saves a copy of the current semantic context (the
3640list of accessible variables) as its semantic value, using alternative
3641@code{context} in the data-type union. Then it calls
3642@code{declare_variable} to add the new variable to that list. Once the
3643first action is finished, the embedded statement @code{stmt} can be
3644parsed. Note that the mid-rule action is component number 5, so the
3645@samp{stmt} is component number 6.
3646
3647After the embedded statement is parsed, its semantic value becomes the
3648value of the entire @code{let}-statement. Then the semantic value from the
3649earlier action is used to restore the prior list of variables. This
3650removes the temporary @code{let}-variable from the list so that it won't
3651appear to exist while the rest of the program is parsed.
3652
3653@findex %destructor
3654@cindex discarded symbols, mid-rule actions
3655@cindex error recovery, mid-rule actions
3656In the above example, if the parser initiates error recovery (@pxref{Error
3657Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3658it might discard the previous semantic context @code{$<context>5} without
3659restoring it.
3660Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3661Discarded Symbols}).
3662However, Bison currently provides no means to declare a destructor specific to
3663a particular mid-rule action's semantic value.
3664
3665One solution is to bury the mid-rule action inside a nonterminal symbol and to
3666declare a destructor for that symbol:
3667
3668@example
3669@group
3670%type <context> let
3671%destructor @{ pop_context ($$); @} let
3672
3673%%
3674
3675stmt: let stmt
3676 @{ $$ = $2;
3677 pop_context ($1); @}
3678 ;
3679
3680let: LET '(' var ')'
3681 @{ $$ = push_context ();
3682 declare_variable ($3); @}
3683 ;
3684
3685@end group
3686@end example
3687
3688@noindent
3689Note that the action is now at the end of its rule.
3690Any mid-rule action can be converted to an end-of-rule action in this way, and
3691this is what Bison actually does to implement mid-rule actions.
3692
3693Taking action before a rule is completely recognized often leads to
3694conflicts since the parser must commit to a parse in order to execute the
3695action. For example, the following two rules, without mid-rule actions,
3696can coexist in a working parser because the parser can shift the open-brace
3697token and look at what follows before deciding whether there is a
3698declaration or not:
3699
3700@example
3701@group
3702compound: '@{' declarations statements '@}'
3703 | '@{' statements '@}'
3704 ;
3705@end group
3706@end example
3707
3708@noindent
3709But when we add a mid-rule action as follows, the rules become nonfunctional:
3710
3711@example
3712@group
3713compound: @{ prepare_for_local_variables (); @}
3714 '@{' declarations statements '@}'
3715@end group
3716@group
3717 | '@{' statements '@}'
3718 ;
3719@end group
3720@end example
3721
3722@noindent
3723Now the parser is forced to decide whether to run the mid-rule action
3724when it has read no farther than the open-brace. In other words, it
3725must commit to using one rule or the other, without sufficient
3726information to do it correctly. (The open-brace token is what is called
3727the @dfn{lookahead} token at this time, since the parser is still
3728deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
3729
3730You might think that you could correct the problem by putting identical
3731actions into the two rules, like this:
3732
3733@example
3734@group
3735compound: @{ prepare_for_local_variables (); @}
3736 '@{' declarations statements '@}'
3737 | @{ prepare_for_local_variables (); @}
3738 '@{' statements '@}'
3739 ;
3740@end group
3741@end example
3742
3743@noindent
3744But this does not help, because Bison does not realize that the two actions
3745are identical. (Bison never tries to understand the C code in an action.)
3746
3747If the grammar is such that a declaration can be distinguished from a
3748statement by the first token (which is true in C), then one solution which
3749does work is to put the action after the open-brace, like this:
3750
3751@example
3752@group
3753compound: '@{' @{ prepare_for_local_variables (); @}
3754 declarations statements '@}'
3755 | '@{' statements '@}'
3756 ;
3757@end group
3758@end example
3759
3760@noindent
3761Now the first token of the following declaration or statement,
3762which would in any case tell Bison which rule to use, can still do so.
3763
3764Another solution is to bury the action inside a nonterminal symbol which
3765serves as a subroutine:
3766
3767@example
3768@group
3769subroutine: /* empty */
3770 @{ prepare_for_local_variables (); @}
3771 ;
3772
3773@end group
3774
3775@group
3776compound: subroutine
3777 '@{' declarations statements '@}'
3778 | subroutine
3779 '@{' statements '@}'
3780 ;
3781@end group
3782@end example
3783
3784@noindent
3785Now Bison can execute the action in the rule for @code{subroutine} without
3786deciding which rule for @code{compound} it will eventually use.
3787
3788@node Named References
3789@subsection Using Named References
3790@cindex named references
3791
3792While every semantic value can be accessed with positional references
3793@code{$@var{n}} and @code{$$}, it's often much more convenient to refer to
3794them by name. First of all, original symbol names may be used as named
3795references. For example:
3796
3797@example
3798@group
3799invocation: op '(' args ')'
3800 @{ $invocation = new_invocation ($op, $args, @@invocation); @}
3801@end group
3802@end example
3803
3804@noindent
3805The positional @code{$$}, @code{@@$}, @code{$n}, and @code{@@n} can be
3806mixed with @code{$name} and @code{@@name} arbitrarily. For example:
3807
3808@example
3809@group
3810invocation: op '(' args ')'
3811 @{ $$ = new_invocation ($op, $args, @@$); @}
3812@end group
3813@end example
3814
3815@noindent
3816However, sometimes regular symbol names are not sufficient due to
3817ambiguities:
3818
3819@example
3820@group
3821exp: exp '/' exp
3822 @{ $exp = $exp / $exp; @} // $exp is ambiguous.
3823
3824exp: exp '/' exp
3825 @{ $$ = $1 / $exp; @} // One usage is ambiguous.
3826
3827exp: exp '/' exp
3828 @{ $$ = $1 / $3; @} // No error.
3829@end group
3830@end example
3831
3832@noindent
3833When ambiguity occurs, explicitly declared names may be used for values and
3834locations. Explicit names are declared as a bracketed name after a symbol
3835appearance in rule definitions. For example:
3836@example
3837@group
3838exp[result]: exp[left] '/' exp[right]
3839 @{ $result = $left / $right; @}
3840@end group
3841@end example
3842
3843@noindent
3844Explicit names may be declared for RHS and for LHS symbols as well. In order
3845to access a semantic value generated by a mid-rule action, an explicit name
3846may also be declared by putting a bracketed name after the closing brace of
3847the mid-rule action code:
3848@example
3849@group
3850exp[res]: exp[x] '+' @{$left = $x;@}[left] exp[right]
3851 @{ $res = $left + $right; @}
3852@end group
3853@end example
3854
3855@noindent
3856
3857In references, in order to specify names containing dots and dashes, an explicit
3858bracketed syntax @code{$[name]} and @code{@@[name]} must be used:
3859@example
3860@group
3861if-stmt: IF '(' expr ')' THEN then.stmt ';'
3862 @{ $[if-stmt] = new_if_stmt ($expr, $[then.stmt]); @}
3863@end group
3864@end example
3865
3866It often happens that named references are followed by a dot, dash or other
3867C punctuation marks and operators. By default, Bison will read
3868@code{$name.suffix} as a reference to symbol value @code{$name} followed by
3869@samp{.suffix}, i.e., an access to the @samp{suffix} field of the semantic
3870value. In order to force Bison to recognize @code{name.suffix} in its entirety
3871as the name of a semantic value, bracketed syntax @code{$[name.suffix]}
3872must be used.
3873
3874
3875@node Locations
3876@section Tracking Locations
3877@cindex location
3878@cindex textual location
3879@cindex location, textual
3880
3881Though grammar rules and semantic actions are enough to write a fully
3882functional parser, it can be useful to process some additional information,
3883especially symbol locations.
3884
3885The way locations are handled is defined by providing a data type, and
3886actions to take when rules are matched.
3887
3888@menu
3889* Location Type:: Specifying a data type for locations.
3890* Actions and Locations:: Using locations in actions.
3891* Location Default Action:: Defining a general way to compute locations.
3892@end menu
3893
3894@node Location Type
3895@subsection Data Type of Locations
3896@cindex data type of locations
3897@cindex default location type
3898
3899Defining a data type for locations is much simpler than for semantic values,
3900since all tokens and groupings always use the same type.
3901
3902You can specify the type of locations by defining a macro called
3903@code{YYLTYPE}, just as you can specify the semantic value type by
3904defining a @code{YYSTYPE} macro (@pxref{Value Type}).
3905When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3906four members:
3907
3908@example
3909typedef struct YYLTYPE
3910@{
3911 int first_line;
3912 int first_column;
3913 int last_line;
3914 int last_column;
3915@} YYLTYPE;
3916@end example
3917
3918When @code{YYLTYPE} is not defined, at the beginning of the parsing, Bison
3919initializes all these fields to 1 for @code{yylloc}. To initialize
3920@code{yylloc} with a custom location type (or to chose a different
3921initialization), use the @code{%initial-action} directive. @xref{Initial
3922Action Decl, , Performing Actions before Parsing}.
3923
3924@node Actions and Locations
3925@subsection Actions and Locations
3926@cindex location actions
3927@cindex actions, location
3928@vindex @@$
3929@vindex @@@var{n}
3930@vindex @@@var{name}
3931@vindex @@[@var{name}]
3932
3933Actions are not only useful for defining language semantics, but also for
3934describing the behavior of the output parser with locations.
3935
3936The most obvious way for building locations of syntactic groupings is very
3937similar to the way semantic values are computed. In a given rule, several
3938constructs can be used to access the locations of the elements being matched.
3939The location of the @var{n}th component of the right hand side is
3940@code{@@@var{n}}, while the location of the left hand side grouping is
3941@code{@@$}.
3942
3943In addition, the named references construct @code{@@@var{name}} and
3944@code{@@[@var{name}]} may also be used to address the symbol locations.
3945@xref{Named References,,Using Named References}, for more information
3946about using the named references construct.
3947
3948Here is a basic example using the default data type for locations:
3949
3950@example
3951@group
3952exp: @dots{}
3953 | exp '/' exp
3954 @{
3955 @@$.first_column = @@1.first_column;
3956 @@$.first_line = @@1.first_line;
3957 @@$.last_column = @@3.last_column;
3958 @@$.last_line = @@3.last_line;
3959 if ($3)
3960 $$ = $1 / $3;
3961 else
3962 @{
3963 $$ = 1;
3964 fprintf (stderr,
3965 "Division by zero, l%d,c%d-l%d,c%d",
3966 @@3.first_line, @@3.first_column,
3967 @@3.last_line, @@3.last_column);
3968 @}
3969 @}
3970@end group
3971@end example
3972
3973As for semantic values, there is a default action for locations that is
3974run each time a rule is matched. It sets the beginning of @code{@@$} to the
3975beginning of the first symbol, and the end of @code{@@$} to the end of the
3976last symbol.
3977
3978With this default action, the location tracking can be fully automatic. The
3979example above simply rewrites this way:
3980
3981@example
3982@group
3983exp: @dots{}
3984 | exp '/' exp
3985 @{
3986 if ($3)
3987 $$ = $1 / $3;
3988 else
3989 @{
3990 $$ = 1;
3991 fprintf (stderr,
3992 "Division by zero, l%d,c%d-l%d,c%d",
3993 @@3.first_line, @@3.first_column,
3994 @@3.last_line, @@3.last_column);
3995 @}
3996 @}
3997@end group
3998@end example
3999
4000@vindex yylloc
4001It is also possible to access the location of the lookahead token, if any,
4002from a semantic action.
4003This location is stored in @code{yylloc}.
4004@xref{Action Features, ,Special Features for Use in Actions}.
4005
4006@node Location Default Action
4007@subsection Default Action for Locations
4008@vindex YYLLOC_DEFAULT
4009@cindex @acronym{GLR} parsers and @code{YYLLOC_DEFAULT}
4010
4011Actually, actions are not the best place to compute locations. Since
4012locations are much more general than semantic values, there is room in
4013the output parser to redefine the default action to take for each
4014rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
4015matched, before the associated action is run. It is also invoked
4016while processing a syntax error, to compute the error's location.
4017Before reporting an unresolvable syntactic ambiguity, a @acronym{GLR}
4018parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
4019of that ambiguity.
4020
4021Most of the time, this macro is general enough to suppress location
4022dedicated code from semantic actions.
4023
4024The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
4025the location of the grouping (the result of the computation). When a
4026rule is matched, the second parameter identifies locations of
4027all right hand side elements of the rule being matched, and the third
4028parameter is the size of the rule's right hand side.
4029When a @acronym{GLR} parser reports an ambiguity, which of multiple candidate
4030right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
4031When processing a syntax error, the second parameter identifies locations
4032of the symbols that were discarded during error processing, and the third
4033parameter is the number of discarded symbols.
4034
4035By default, @code{YYLLOC_DEFAULT} is defined this way:
4036
4037@smallexample
4038@group
4039# define YYLLOC_DEFAULT(Current, Rhs, N) \
4040 do \
4041 if (N) \
4042 @{ \
4043 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
4044 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
4045 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
4046 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
4047 @} \
4048 else \
4049 @{ \
4050 (Current).first_line = (Current).last_line = \
4051 YYRHSLOC(Rhs, 0).last_line; \
4052 (Current).first_column = (Current).last_column = \
4053 YYRHSLOC(Rhs, 0).last_column; \
4054 @} \
4055 while (0)
4056@end group
4057@end smallexample
4058
4059where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
4060in @var{rhs} when @var{k} is positive, and the location of the symbol
4061just before the reduction when @var{k} and @var{n} are both zero.
4062
4063When defining @code{YYLLOC_DEFAULT}, you should consider that:
4064
4065@itemize @bullet
4066@item
4067All arguments are free of side-effects. However, only the first one (the
4068result) should be modified by @code{YYLLOC_DEFAULT}.
4069
4070@item
4071For consistency with semantic actions, valid indexes within the
4072right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
4073valid index, and it refers to the symbol just before the reduction.
4074During error processing @var{n} is always positive.
4075
4076@item
4077Your macro should parenthesize its arguments, if need be, since the
4078actual arguments may not be surrounded by parentheses. Also, your
4079macro should expand to something that can be used as a single
4080statement when it is followed by a semicolon.
4081@end itemize
4082
4083@node Declarations
4084@section Bison Declarations
4085@cindex declarations, Bison
4086@cindex Bison declarations
4087
4088The @dfn{Bison declarations} section of a Bison grammar defines the symbols
4089used in formulating the grammar and the data types of semantic values.
4090@xref{Symbols}.
4091
4092All token type names (but not single-character literal tokens such as
4093@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
4094declared if you need to specify which data type to use for the semantic
4095value (@pxref{Multiple Types, ,More Than One Value Type}).
4096
4097The first rule in the file also specifies the start symbol, by default.
4098If you want some other symbol to be the start symbol, you must declare
4099it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
4100Grammars}).
4101
4102@menu
4103* Require Decl:: Requiring a Bison version.
4104* Token Decl:: Declaring terminal symbols.
4105* Precedence Decl:: Declaring terminals with precedence and associativity.
4106* Union Decl:: Declaring the set of all semantic value types.
4107* Type Decl:: Declaring the choice of type for a nonterminal symbol.
4108* Initial Action Decl:: Code run before parsing starts.
4109* Destructor Decl:: Declaring how symbols are freed.
4110* Expect Decl:: Suppressing warnings about parsing conflicts.
4111* Start Decl:: Specifying the start symbol.
4112* Pure Decl:: Requesting a reentrant parser.
4113* Push Decl:: Requesting a push parser.
4114* Decl Summary:: Table of all Bison declarations.
4115@end menu
4116
4117@node Require Decl
4118@subsection Require a Version of Bison
4119@cindex version requirement
4120@cindex requiring a version of Bison
4121@findex %require
4122
4123You may require the minimum version of Bison to process the grammar. If
4124the requirement is not met, @command{bison} exits with an error (exit
4125status 63).
4126
4127@example
4128%require "@var{version}"
4129@end example
4130
4131@node Token Decl
4132@subsection Token Type Names
4133@cindex declaring token type names
4134@cindex token type names, declaring
4135@cindex declaring literal string tokens
4136@findex %token
4137
4138The basic way to declare a token type name (terminal symbol) is as follows:
4139
4140@example
4141%token @var{name}
4142@end example
4143
4144Bison will convert this into a @code{#define} directive in
4145the parser, so that the function @code{yylex} (if it is in this file)
4146can use the name @var{name} to stand for this token type's code.
4147
4148Alternatively, you can use @code{%left}, @code{%right},
4149@code{%precedence}, or
4150@code{%nonassoc} instead of @code{%token}, if you wish to specify
4151associativity and precedence. @xref{Precedence Decl, ,Operator
4152Precedence}.
4153
4154You can explicitly specify the numeric code for a token type by appending
4155a nonnegative decimal or hexadecimal integer value in the field immediately
4156following the token name:
4157
4158@example
4159%token NUM 300
4160%token XNUM 0x12d // a GNU extension
4161@end example
4162
4163@noindent
4164It is generally best, however, to let Bison choose the numeric codes for
4165all token types. Bison will automatically select codes that don't conflict
4166with each other or with normal characters.
4167
4168In the event that the stack type is a union, you must augment the
4169@code{%token} or other token declaration to include the data type
4170alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4171Than One Value Type}).
4172
4173For example:
4174
4175@example
4176@group
4177%union @{ /* define stack type */
4178 double val;
4179 symrec *tptr;
4180@}
4181%token <val> NUM /* define token NUM and its type */
4182@end group
4183@end example
4184
4185You can associate a literal string token with a token type name by
4186writing the literal string at the end of a @code{%token}
4187declaration which declares the name. For example:
4188
4189@example
4190%token arrow "=>"
4191@end example
4192
4193@noindent
4194For example, a grammar for the C language might specify these names with
4195equivalent literal string tokens:
4196
4197@example
4198%token <operator> OR "||"
4199%token <operator> LE 134 "<="
4200%left OR "<="
4201@end example
4202
4203@noindent
4204Once you equate the literal string and the token name, you can use them
4205interchangeably in further declarations or the grammar rules. The
4206@code{yylex} function can use the token name or the literal string to
4207obtain the token type code number (@pxref{Calling Convention}).
4208Syntax error messages passed to @code{yyerror} from the parser will reference
4209the literal string instead of the token name.
4210
4211The token numbered as 0 corresponds to end of file; the following line
4212allows for nicer error messages referring to ``end of file'' instead
4213of ``$end'':
4214
4215@example
4216%token END 0 "end of file"
4217@end example
4218
4219@node Precedence Decl
4220@subsection Operator Precedence
4221@cindex precedence declarations
4222@cindex declaring operator precedence
4223@cindex operator precedence, declaring
4224
4225Use the @code{%left}, @code{%right}, @code{%nonassoc}, or
4226@code{%precedence} declaration to
4227declare a token and specify its precedence and associativity, all at
4228once. These are called @dfn{precedence declarations}.
4229@xref{Precedence, ,Operator Precedence}, for general information on
4230operator precedence.
4231
4232The syntax of a precedence declaration is nearly the same as that of
4233@code{%token}: either
4234
4235@example
4236%left @var{symbols}@dots{}
4237@end example
4238
4239@noindent
4240or
4241
4242@example
4243%left <@var{type}> @var{symbols}@dots{}
4244@end example
4245
4246And indeed any of these declarations serves the purposes of @code{%token}.
4247But in addition, they specify the associativity and relative precedence for
4248all the @var{symbols}:
4249
4250@itemize @bullet
4251@item
4252The associativity of an operator @var{op} determines how repeated uses
4253of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4254@var{z}} is parsed by grouping @var{x} with @var{y} first or by
4255grouping @var{y} with @var{z} first. @code{%left} specifies
4256left-associativity (grouping @var{x} with @var{y} first) and
4257@code{%right} specifies right-associativity (grouping @var{y} with
4258@var{z} first). @code{%nonassoc} specifies no associativity, which
4259means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4260considered a syntax error.
4261
4262@code{%precedence} gives only precedence to the @var{symbols}, and
4263defines no associativity at all. Use this to define precedence only,
4264and leave any potential conflict due to associativity enabled.
4265
4266@item
4267The precedence of an operator determines how it nests with other operators.
4268All the tokens declared in a single precedence declaration have equal
4269precedence and nest together according to their associativity.
4270When two tokens declared in different precedence declarations associate,
4271the one declared later has the higher precedence and is grouped first.
4272@end itemize
4273
4274For backward compatibility, there is a confusing difference between the
4275argument lists of @code{%token} and precedence declarations.
4276Only a @code{%token} can associate a literal string with a token type name.
4277A precedence declaration always interprets a literal string as a reference to a
4278separate token.
4279For example:
4280
4281@example
4282%left OR "<=" // Does not declare an alias.
4283%left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4284@end example
4285
4286@node Union Decl
4287@subsection The Collection of Value Types
4288@cindex declaring value types
4289@cindex value types, declaring
4290@findex %union
4291
4292The @code{%union} declaration specifies the entire collection of
4293possible data types for semantic values. The keyword @code{%union} is
4294followed by braced code containing the same thing that goes inside a
4295@code{union} in C@.
4296
4297For example:
4298
4299@example
4300@group
4301%union @{
4302 double val;
4303 symrec *tptr;
4304@}
4305@end group
4306@end example
4307
4308@noindent
4309This says that the two alternative types are @code{double} and @code{symrec
4310*}. They are given names @code{val} and @code{tptr}; these names are used
4311in the @code{%token} and @code{%type} declarations to pick one of the types
4312for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4313
4314As an extension to @acronym{POSIX}, a tag is allowed after the
4315@code{union}. For example:
4316
4317@example
4318@group
4319%union value @{
4320 double val;
4321 symrec *tptr;
4322@}
4323@end group
4324@end example
4325
4326@noindent
4327specifies the union tag @code{value}, so the corresponding C type is
4328@code{union value}. If you do not specify a tag, it defaults to
4329@code{YYSTYPE}.
4330
4331As another extension to @acronym{POSIX}, you may specify multiple
4332@code{%union} declarations; their contents are concatenated. However,
4333only the first @code{%union} declaration can specify a tag.
4334
4335Note that, unlike making a @code{union} declaration in C, you need not write
4336a semicolon after the closing brace.
4337
4338Instead of @code{%union}, you can define and use your own union type
4339@code{YYSTYPE} if your grammar contains at least one
4340@samp{<@var{type}>} tag. For example, you can put the following into
4341a header file @file{parser.h}:
4342
4343@example
4344@group
4345union YYSTYPE @{
4346 double val;
4347 symrec *tptr;
4348@};
4349typedef union YYSTYPE YYSTYPE;
4350@end group
4351@end example
4352
4353@noindent
4354and then your grammar can use the following
4355instead of @code{%union}:
4356
4357@example
4358@group
4359%@{
4360#include "parser.h"
4361%@}
4362%type <val> expr
4363%token <tptr> ID
4364@end group
4365@end example
4366
4367@node Type Decl
4368@subsection Nonterminal Symbols
4369@cindex declaring value types, nonterminals
4370@cindex value types, nonterminals, declaring
4371@findex %type
4372
4373@noindent
4374When you use @code{%union} to specify multiple value types, you must
4375declare the value type of each nonterminal symbol for which values are
4376used. This is done with a @code{%type} declaration, like this:
4377
4378@example
4379%type <@var{type}> @var{nonterminal}@dots{}
4380@end example
4381
4382@noindent
4383Here @var{nonterminal} is the name of a nonterminal symbol, and
4384@var{type} is the name given in the @code{%union} to the alternative
4385that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4386can give any number of nonterminal symbols in the same @code{%type}
4387declaration, if they have the same value type. Use spaces to separate
4388the symbol names.
4389
4390You can also declare the value type of a terminal symbol. To do this,
4391use the same @code{<@var{type}>} construction in a declaration for the
4392terminal symbol. All kinds of token declarations allow
4393@code{<@var{type}>}.
4394
4395@node Initial Action Decl
4396@subsection Performing Actions before Parsing
4397@findex %initial-action
4398
4399Sometimes your parser needs to perform some initializations before
4400parsing. The @code{%initial-action} directive allows for such arbitrary
4401code.
4402
4403@deffn {Directive} %initial-action @{ @var{code} @}
4404@findex %initial-action
4405Declare that the braced @var{code} must be invoked before parsing each time
4406@code{yyparse} is called. The @var{code} may use @code{$$} and
4407@code{@@$} --- initial value and location of the lookahead --- and the
4408@code{%parse-param}.
4409@end deffn
4410
4411For instance, if your locations use a file name, you may use
4412
4413@example
4414%parse-param @{ char const *file_name @};
4415%initial-action
4416@{
4417 @@$.initialize (file_name);
4418@};
4419@end example
4420
4421
4422@node Destructor Decl
4423@subsection Freeing Discarded Symbols
4424@cindex freeing discarded symbols
4425@findex %destructor
4426@findex <*>
4427@findex <>
4428During error recovery (@pxref{Error Recovery}), symbols already pushed
4429on the stack and tokens coming from the rest of the file are discarded
4430until the parser falls on its feet. If the parser runs out of memory,
4431or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
4432symbols on the stack must be discarded. Even if the parser succeeds, it
4433must discard the start symbol.
4434
4435When discarded symbols convey heap based information, this memory is
4436lost. While this behavior can be tolerable for batch parsers, such as
4437in traditional compilers, it is unacceptable for programs like shells or
4438protocol implementations that may parse and execute indefinitely.
4439
4440The @code{%destructor} directive defines code that is called when a
4441symbol is automatically discarded.
4442
4443@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4444@findex %destructor
4445Invoke the braced @var{code} whenever the parser discards one of the
4446@var{symbols}.
4447Within @var{code}, @code{$$} designates the semantic value associated
4448with the discarded symbol, and @code{@@$} designates its location.
4449The additional parser parameters are also available (@pxref{Parser Function, ,
4450The Parser Function @code{yyparse}}).
4451
4452When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4453per-symbol @code{%destructor}.
4454You may also define a per-type @code{%destructor} by listing a semantic type
4455tag among @var{symbols}.
4456In that case, the parser will invoke this @var{code} whenever it discards any
4457grammar symbol that has that semantic type tag unless that symbol has its own
4458per-symbol @code{%destructor}.
4459
4460Finally, you can define two different kinds of default @code{%destructor}s.
4461(These default forms are experimental.
4462More user feedback will help to determine whether they should become permanent
4463features.)
4464You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
4465exactly one @code{%destructor} declaration in your grammar file.
4466The parser will invoke the @var{code} associated with one of these whenever it
4467discards any user-defined grammar symbol that has no per-symbol and no per-type
4468@code{%destructor}.
4469The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4470symbol for which you have formally declared a semantic type tag (@code{%type}
4471counts as such a declaration, but @code{$<tag>$} does not).
4472The parser uses the @var{code} for @code{<>} in the case of such a grammar
4473symbol that has no declared semantic type tag.
4474@end deffn
4475
4476@noindent
4477For example:
4478
4479@smallexample
4480%union @{ char *string; @}
4481%token <string> STRING1
4482%token <string> STRING2
4483%type <string> string1
4484%type <string> string2
4485%union @{ char character; @}
4486%token <character> CHR
4487%type <character> chr
4488%token TAGLESS
4489
4490%destructor @{ @} <character>
4491%destructor @{ free ($$); @} <*>
4492%destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
4493%destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
4494@end smallexample
4495
4496@noindent
4497guarantees that, when the parser discards any user-defined symbol that has a
4498semantic type tag other than @code{<character>}, it passes its semantic value
4499to @code{free} by default.
4500However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4501prints its line number to @code{stdout}.
4502It performs only the second @code{%destructor} in this case, so it invokes
4503@code{free} only once.
4504Finally, the parser merely prints a message whenever it discards any symbol,
4505such as @code{TAGLESS}, that has no semantic type tag.
4506
4507A Bison-generated parser invokes the default @code{%destructor}s only for
4508user-defined as opposed to Bison-defined symbols.
4509For example, the parser will not invoke either kind of default
4510@code{%destructor} for the special Bison-defined symbols @code{$accept},
4511@code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4512none of which you can reference in your grammar.
4513It also will not invoke either for the @code{error} token (@pxref{Table of
4514Symbols, ,error}), which is always defined by Bison regardless of whether you
4515reference it in your grammar.
4516However, it may invoke one of them for the end token (token 0) if you
4517redefine it from @code{$end} to, for example, @code{END}:
4518
4519@smallexample
4520%token END 0
4521@end smallexample
4522
4523@cindex actions in mid-rule
4524@cindex mid-rule actions
4525Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4526mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4527That is, Bison does not consider a mid-rule to have a semantic value if you do
4528not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4529@var{n} is the RHS symbol position of the mid-rule) in any later action in that
4530rule.
4531However, if you do reference either, the Bison-generated parser will invoke the
4532@code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
4533
4534@ignore
4535@noindent
4536In the future, it may be possible to redefine the @code{error} token as a
4537nonterminal that captures the discarded symbols.
4538In that case, the parser will invoke the default destructor for it as well.
4539@end ignore
4540
4541@sp 1
4542
4543@cindex discarded symbols
4544@dfn{Discarded symbols} are the following:
4545
4546@itemize
4547@item
4548stacked symbols popped during the first phase of error recovery,
4549@item
4550incoming terminals during the second phase of error recovery,
4551@item
4552the current lookahead and the entire stack (except the current
4553right-hand side symbols) when the parser returns immediately, and
4554@item
4555the start symbol, when the parser succeeds.
4556@end itemize
4557
4558The parser can @dfn{return immediately} because of an explicit call to
4559@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4560exhaustion.
4561
4562Right-hand side symbols of a rule that explicitly triggers a syntax
4563error via @code{YYERROR} are not discarded automatically. As a rule
4564of thumb, destructors are invoked only when user actions cannot manage
4565the memory.
4566
4567@node Expect Decl
4568@subsection Suppressing Conflict Warnings
4569@cindex suppressing conflict warnings
4570@cindex preventing warnings about conflicts
4571@cindex warnings, preventing
4572@cindex conflicts, suppressing warnings of
4573@findex %expect
4574@findex %expect-rr
4575
4576Bison normally warns if there are any conflicts in the grammar
4577(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4578have harmless shift/reduce conflicts which are resolved in a predictable
4579way and would be difficult to eliminate. It is desirable to suppress
4580the warning about these conflicts unless the number of conflicts
4581changes. You can do this with the @code{%expect} declaration.
4582
4583The declaration looks like this:
4584
4585@example
4586%expect @var{n}
4587@end example
4588
4589Here @var{n} is a decimal integer. The declaration says there should
4590be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4591Bison reports an error if the number of shift/reduce conflicts differs
4592from @var{n}, or if there are any reduce/reduce conflicts.
4593
4594For deterministic parsers, reduce/reduce conflicts are more
4595serious, and should be eliminated entirely. Bison will always report
4596reduce/reduce conflicts for these parsers. With @acronym{GLR}
4597parsers, however, both kinds of conflicts are routine; otherwise,
4598there would be no need to use @acronym{GLR} parsing. Therefore, it is
4599also possible to specify an expected number of reduce/reduce conflicts
4600in @acronym{GLR} parsers, using the declaration:
4601
4602@example
4603%expect-rr @var{n}
4604@end example
4605
4606In general, using @code{%expect} involves these steps:
4607
4608@itemize @bullet
4609@item
4610Compile your grammar without @code{%expect}. Use the @samp{-v} option
4611to get a verbose list of where the conflicts occur. Bison will also
4612print the number of conflicts.
4613
4614@item
4615Check each of the conflicts to make sure that Bison's default
4616resolution is what you really want. If not, rewrite the grammar and
4617go back to the beginning.
4618
4619@item
4620Add an @code{%expect} declaration, copying the number @var{n} from the
4621number which Bison printed. With @acronym{GLR} parsers, add an
4622@code{%expect-rr} declaration as well.
4623@end itemize
4624
4625Now Bison will warn you if you introduce an unexpected conflict, but
4626will keep silent otherwise.
4627
4628@node Start Decl
4629@subsection The Start-Symbol
4630@cindex declaring the start symbol
4631@cindex start symbol, declaring
4632@cindex default start symbol
4633@findex %start
4634
4635Bison assumes by default that the start symbol for the grammar is the first
4636nonterminal specified in the grammar specification section. The programmer
4637may override this restriction with the @code{%start} declaration as follows:
4638
4639@example
4640%start @var{symbol}
4641@end example
4642
4643@node Pure Decl
4644@subsection A Pure (Reentrant) Parser
4645@cindex reentrant parser
4646@cindex pure parser
4647@findex %define api.pure
4648
4649A @dfn{reentrant} program is one which does not alter in the course of
4650execution; in other words, it consists entirely of @dfn{pure} (read-only)
4651code. Reentrancy is important whenever asynchronous execution is possible;
4652for example, a nonreentrant program may not be safe to call from a signal
4653handler. In systems with multiple threads of control, a nonreentrant
4654program must be called only within interlocks.
4655
4656Normally, Bison generates a parser which is not reentrant. This is
4657suitable for most uses, and it permits compatibility with Yacc. (The
4658standard Yacc interfaces are inherently nonreentrant, because they use
4659statically allocated variables for communication with @code{yylex},
4660including @code{yylval} and @code{yylloc}.)
4661
4662Alternatively, you can generate a pure, reentrant parser. The Bison
4663declaration @samp{%define api.pure} says that you want the parser to be
4664reentrant. It looks like this:
4665
4666@example
4667%define api.pure
4668@end example
4669
4670The result is that the communication variables @code{yylval} and
4671@code{yylloc} become local variables in @code{yyparse}, and a different
4672calling convention is used for the lexical analyzer function
4673@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4674Parsers}, for the details of this. The variable @code{yynerrs}
4675becomes local in @code{yyparse} in pull mode but it becomes a member
4676of yypstate in push mode. (@pxref{Error Reporting, ,The Error
4677Reporting Function @code{yyerror}}). The convention for calling
4678@code{yyparse} itself is unchanged.
4679
4680Whether the parser is pure has nothing to do with the grammar rules.
4681You can generate either a pure parser or a nonreentrant parser from any
4682valid grammar.
4683
4684@node Push Decl
4685@subsection A Push Parser
4686@cindex push parser
4687@cindex push parser
4688@findex %define api.push-pull
4689
4690(The current push parsing interface is experimental and may evolve.
4691More user feedback will help to stabilize it.)
4692
4693A pull parser is called once and it takes control until all its input
4694is completely parsed. A push parser, on the other hand, is called
4695each time a new token is made available.
4696
4697A push parser is typically useful when the parser is part of a
4698main event loop in the client's application. This is typically
4699a requirement of a GUI, when the main event loop needs to be triggered
4700within a certain time period.
4701
4702Normally, Bison generates a pull parser.
4703The following Bison declaration says that you want the parser to be a push
4704parser (@pxref{Decl Summary,,%define api.push-pull}):
4705
4706@example
4707%define api.push-pull push
4708@end example
4709
4710In almost all cases, you want to ensure that your push parser is also
4711a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
4712time you should create an impure push parser is to have backwards
4713compatibility with the impure Yacc pull mode interface. Unless you know
4714what you are doing, your declarations should look like this:
4715
4716@example
4717%define api.pure
4718%define api.push-pull push
4719@end example
4720
4721There is a major notable functional difference between the pure push parser
4722and the impure push parser. It is acceptable for a pure push parser to have
4723many parser instances, of the same type of parser, in memory at the same time.
4724An impure push parser should only use one parser at a time.
4725
4726When a push parser is selected, Bison will generate some new symbols in
4727the generated parser. @code{yypstate} is a structure that the generated
4728parser uses to store the parser's state. @code{yypstate_new} is the
4729function that will create a new parser instance. @code{yypstate_delete}
4730will free the resources associated with the corresponding parser instance.
4731Finally, @code{yypush_parse} is the function that should be called whenever a
4732token is available to provide the parser. A trivial example
4733of using a pure push parser would look like this:
4734
4735@example
4736int status;
4737yypstate *ps = yypstate_new ();
4738do @{
4739 status = yypush_parse (ps, yylex (), NULL);
4740@} while (status == YYPUSH_MORE);
4741yypstate_delete (ps);
4742@end example
4743
4744If the user decided to use an impure push parser, a few things about
4745the generated parser will change. The @code{yychar} variable becomes
4746a global variable instead of a variable in the @code{yypush_parse} function.
4747For this reason, the signature of the @code{yypush_parse} function is
4748changed to remove the token as a parameter. A nonreentrant push parser
4749example would thus look like this:
4750
4751@example
4752extern int yychar;
4753int status;
4754yypstate *ps = yypstate_new ();
4755do @{
4756 yychar = yylex ();
4757 status = yypush_parse (ps);
4758@} while (status == YYPUSH_MORE);
4759yypstate_delete (ps);
4760@end example
4761
4762That's it. Notice the next token is put into the global variable @code{yychar}
4763for use by the next invocation of the @code{yypush_parse} function.
4764
4765Bison also supports both the push parser interface along with the pull parser
4766interface in the same generated parser. In order to get this functionality,
4767you should replace the @samp{%define api.push-pull push} declaration with the
4768@samp{%define api.push-pull both} declaration. Doing this will create all of
4769the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
4770and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4771would be used. However, the user should note that it is implemented in the
4772generated parser by calling @code{yypull_parse}.
4773This makes the @code{yyparse} function that is generated with the
4774@samp{%define api.push-pull both} declaration slower than the normal
4775@code{yyparse} function. If the user
4776calls the @code{yypull_parse} function it will parse the rest of the input
4777stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4778and then @code{yypull_parse} the rest of the input stream. If you would like
4779to switch back and forth between between parsing styles, you would have to
4780write your own @code{yypull_parse} function that knows when to quit looking
4781for input. An example of using the @code{yypull_parse} function would look
4782like this:
4783
4784@example
4785yypstate *ps = yypstate_new ();
4786yypull_parse (ps); /* Will call the lexer */
4787yypstate_delete (ps);
4788@end example
4789
4790Adding the @samp{%define api.pure} declaration does exactly the same thing to
4791the generated parser with @samp{%define api.push-pull both} as it did for
4792@samp{%define api.push-pull push}.
4793
4794@node Decl Summary
4795@subsection Bison Declaration Summary
4796@cindex Bison declaration summary
4797@cindex declaration summary
4798@cindex summary, Bison declaration
4799
4800Here is a summary of the declarations used to define a grammar:
4801
4802@deffn {Directive} %union
4803Declare the collection of data types that semantic values may have
4804(@pxref{Union Decl, ,The Collection of Value Types}).
4805@end deffn
4806
4807@deffn {Directive} %token
4808Declare a terminal symbol (token type name) with no precedence
4809or associativity specified (@pxref{Token Decl, ,Token Type Names}).
4810@end deffn
4811
4812@deffn {Directive} %right
4813Declare a terminal symbol (token type name) that is right-associative
4814(@pxref{Precedence Decl, ,Operator Precedence}).
4815@end deffn
4816
4817@deffn {Directive} %left
4818Declare a terminal symbol (token type name) that is left-associative
4819(@pxref{Precedence Decl, ,Operator Precedence}).
4820@end deffn
4821
4822@deffn {Directive} %nonassoc
4823Declare a terminal symbol (token type name) that is nonassociative
4824(@pxref{Precedence Decl, ,Operator Precedence}).
4825Using it in a way that would be associative is a syntax error.
4826@end deffn
4827
4828@ifset defaultprec
4829@deffn {Directive} %default-prec
4830Assign a precedence to rules lacking an explicit @code{%prec} modifier
4831(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4832@end deffn
4833@end ifset
4834
4835@deffn {Directive} %type
4836Declare the type of semantic values for a nonterminal symbol
4837(@pxref{Type Decl, ,Nonterminal Symbols}).
4838@end deffn
4839
4840@deffn {Directive} %start
4841Specify the grammar's start symbol (@pxref{Start Decl, ,The
4842Start-Symbol}).
4843@end deffn
4844
4845@deffn {Directive} %expect
4846Declare the expected number of shift-reduce conflicts
4847(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
4848@end deffn
4849
4850
4851@sp 1
4852@noindent
4853In order to change the behavior of @command{bison}, use the following
4854directives:
4855
4856@deffn {Directive} %code @{@var{code}@}
4857@findex %code
4858This is the unqualified form of the @code{%code} directive.
4859It inserts @var{code} verbatim at a language-dependent default location in the
4860output@footnote{The default location is actually skeleton-dependent;
4861 writers of non-standard skeletons however should choose the default location
4862 consistently with the behavior of the standard Bison skeletons.}.
4863
4864@cindex Prologue
4865For C/C++, the default location is the parser source code
4866file after the usual contents of the parser header file.
4867Thus, @code{%code} replaces the traditional Yacc prologue,
4868@code{%@{@var{code}%@}}, for most purposes.
4869For a detailed discussion, see @ref{Prologue Alternatives}.
4870
4871For Java, the default location is inside the parser class.
4872@end deffn
4873
4874@deffn {Directive} %code @var{qualifier} @{@var{code}@}
4875This is the qualified form of the @code{%code} directive.
4876If you need to specify location-sensitive verbatim @var{code} that does not
4877belong at the default location selected by the unqualified @code{%code} form,
4878use this form instead.
4879
4880@var{qualifier} identifies the purpose of @var{code} and thus the location(s)
4881where Bison should generate it.
4882Not all @var{qualifier}s are accepted for all target languages.
4883Unaccepted @var{qualifier}s produce an error.
4884Some of the accepted @var{qualifier}s are:
4885
4886@itemize @bullet
4887@item requires
4888@findex %code requires
4889
4890@itemize @bullet
4891@item Language(s): C, C++
4892
4893@item Purpose: This is the best place to write dependency code required for
4894@code{YYSTYPE} and @code{YYLTYPE}.
4895In other words, it's the best place to define types referenced in @code{%union}
4896directives, and it's the best place to override Bison's default @code{YYSTYPE}
4897and @code{YYLTYPE} definitions.
4898
4899@item Location(s): The parser header file and the parser source code file
4900before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE} definitions.
4901@end itemize
4902
4903@item provides
4904@findex %code provides
4905
4906@itemize @bullet
4907@item Language(s): C, C++
4908
4909@item Purpose: This is the best place to write additional definitions and
4910declarations that should be provided to other modules.
4911
4912@item Location(s): The parser header file and the parser source code file after
4913the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and token definitions.
4914@end itemize
4915
4916@item top
4917@findex %code top
4918
4919@itemize @bullet
4920@item Language(s): C, C++
4921
4922@item Purpose: The unqualified @code{%code} or @code{%code requires} should
4923usually be more appropriate than @code{%code top}.
4924However, occasionally it is necessary to insert code much nearer the top of the
4925parser source code file.
4926For example:
4927
4928@smallexample
4929%code top @{
4930 #define _GNU_SOURCE
4931 #include <stdio.h>
4932@}
4933@end smallexample
4934
4935@item Location(s): Near the top of the parser source code file.
4936@end itemize
4937
4938@item imports
4939@findex %code imports
4940
4941@itemize @bullet
4942@item Language(s): Java
4943
4944@item Purpose: This is the best place to write Java import directives.
4945
4946@item Location(s): The parser Java file after any Java package directive and
4947before any class definitions.
4948@end itemize
4949@end itemize
4950
4951@cindex Prologue
4952For a detailed discussion of how to use @code{%code} in place of the
4953traditional Yacc prologue for C/C++, see @ref{Prologue Alternatives}.
4954@end deffn
4955
4956@deffn {Directive} %debug
4957Instrument the output parser for traces. Obsoleted by @samp{%define
4958parse.trace}.
4959@xref{Tracing, ,Tracing Your Parser}.
4960@end deffn
4961
4962@deffn {Directive} %define @var{variable}
4963@deffnx {Directive} %define @var{variable} @var{value}
4964@deffnx {Directive} %define @var{variable} "@var{value}"
4965Define a variable to adjust Bison's behavior.
4966
4967It is an error if a @var{variable} is defined by @code{%define} multiple
4968times, but see @ref{Bison Options,,-D @var{name}[=@var{value}]}.
4969
4970@var{value} must be placed in quotation marks if it contains any
4971character other than a letter, underscore, period, dash, or non-initial
4972digit.
4973
4974Omitting @code{"@var{value}"} entirely is always equivalent to specifying
4975@code{""}.
4976
4977Some @var{variable}s take Boolean values.
4978In this case, Bison will complain if the variable definition does not meet one
4979of the following four conditions:
4980
4981@enumerate
4982@item @code{@var{value}} is @code{true}
4983
4984@item @code{@var{value}} is omitted (or @code{""} is specified).
4985This is equivalent to @code{true}.
4986
4987@item @code{@var{value}} is @code{false}.
4988
4989@item @var{variable} is never defined.
4990In this case, Bison selects a default value.
4991@end enumerate
4992
4993What @var{variable}s are accepted, as well as their meanings and default
4994values, depend on the selected target language and/or the parser
4995skeleton (@pxref{Decl Summary,,%language}, @pxref{Decl
4996Summary,,%skeleton}).
4997Unaccepted @var{variable}s produce an error.
4998Some of the accepted @var{variable}s are:
4999
5000@table @code
5001@c ================================================== api.namespace
5002@item api.namespace
5003@findex %define api.namespace
5004@itemize
5005@item Languages(s): C++
5006
5007@item Purpose: Specifies the namespace for the parser class.
5008For example, if you specify:
5009
5010@smallexample
5011%define api.namespace "foo::bar"
5012@end smallexample
5013
5014Bison uses @code{foo::bar} verbatim in references such as:
5015
5016@smallexample
5017foo::bar::parser::semantic_type
5018@end smallexample
5019
5020However, to open a namespace, Bison removes any leading @code{::} and then
5021splits on any remaining occurrences:
5022
5023@smallexample
5024namespace foo @{ namespace bar @{
5025 class position;
5026 class location;
5027@} @}
5028@end smallexample
5029
5030@item Accepted Values:
5031Any absolute or relative C++ namespace reference without a trailing
5032@code{"::"}. For example, @code{"foo"} or @code{"::foo::bar"}.
5033
5034@item Default Value:
5035The value specified by @code{%name-prefix}, which defaults to @code{yy}.
5036This usage of @code{%name-prefix} is for backward compatibility and can
5037be confusing since @code{%name-prefix} also specifies the textual prefix
5038for the lexical analyzer function. Thus, if you specify
5039@code{%name-prefix}, it is best to also specify @samp{%define
5040api.namespace} so that @code{%name-prefix} @emph{only} affects the
5041lexical analyzer function. For example, if you specify:
5042
5043@smallexample
5044%define api.namespace "foo"
5045%name-prefix "bar::"
5046@end smallexample
5047
5048The parser namespace is @code{foo} and @code{yylex} is referenced as
5049@code{bar::lex}.
5050@end itemize
5051@c namespace
5052
5053
5054
5055@c ================================================== api.pure
5056@item api.pure
5057@findex %define api.pure
5058
5059@itemize @bullet
5060@item Language(s): C
5061
5062@item Purpose: Request a pure (reentrant) parser program.
5063@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
5064
5065@item Accepted Values: Boolean
5066
5067@item Default Value: @code{false}
5068@end itemize
5069@c api.pure
5070
5071
5072
5073@c ================================================== api.push-pull
5074@item api.push-pull
5075@findex %define api.push-pull
5076
5077@itemize @bullet
5078@item Language(s): C (deterministic parsers only)
5079
5080@item Purpose: Requests a pull parser, a push parser, or both.
5081@xref{Push Decl, ,A Push Parser}.
5082(The current push parsing interface is experimental and may evolve.
5083More user feedback will help to stabilize it.)
5084
5085@item Accepted Values: @code{pull}, @code{push}, @code{both}
5086
5087@item Default Value: @code{pull}
5088@end itemize
5089@c api.push-pull
5090
5091
5092
5093@c ================================================== api.tokens.prefix
5094@item api.tokens.prefix
5095@findex %define api.tokens.prefix
5096
5097@itemize
5098@item Languages(s): all
5099
5100@item Purpose:
5101Add a prefix to the token names when generating their definition in the
5102target language. For instance
5103
5104@example
5105%token FILE for ERROR
5106%define api.tokens.prefix "TOK_"
5107%%
5108start: FILE for ERROR;
5109@end example
5110
5111@noindent
5112generates the definition of the symbols @code{TOK_FILE}, @code{TOK_for},
5113and @code{TOK_ERROR} in the generated source files. In particular, the
5114scanner must use these prefixed token names, while the grammar itself
5115may still use the short names (as in the sample rule given above). The
5116generated informational files (@file{*.output}, @file{*.xml},
5117@file{*.dot}) are not modified by this prefix. See @ref{Calc++ Parser}
5118and @ref{Calc++ Scanner}, for a complete example.
5119
5120@item Accepted Values:
5121Any string. Should be a valid identifier prefix in the target language,
5122in other words, it should typically be an identifier itself (sequence of
5123letters, underscores, and ---not at the beginning--- digits).
5124
5125@item Default Value:
5126empty
5127@end itemize
5128@c api.tokens.prefix
5129
5130
5131@c ================================================== lex_symbol
5132@item variant
5133@findex %define lex_symbol
5134
5135@itemize @bullet
5136@item Language(s):
5137C++
5138
5139@item Purpose:
5140When variant-based semantic values are enabled (@pxref{C++ Variants}),
5141request that symbols be handled as a whole (type, value, and possibly
5142location) in the scanner. @xref{Complete Symbols}, for details.
5143
5144@item Accepted Values:
5145Boolean.
5146
5147@item Default Value:
5148@code{false}
5149@end itemize
5150@c lex_symbol
5151
5152
5153@c ================================================== lr.default-reductions
5154
5155@item lr.default-reductions
5156@cindex default reductions
5157@findex %define lr.default-reductions
5158@cindex delayed syntax errors
5159@cindex syntax errors delayed
5160
5161@itemize @bullet
5162@item Language(s): all
5163
5164@item Purpose: Specifies the kind of states that are permitted to
5165contain default reductions.
5166That is, in such a state, Bison declares the reduction with the largest
5167lookahead set to be the default reduction and then removes that
5168lookahead set.
5169The advantages of default reductions are discussed below.
5170The disadvantage is that, when the generated parser encounters a
5171syntactically unacceptable token, the parser might then perform
5172unnecessary default reductions before it can detect the syntax error.
5173
5174(This feature is experimental.
5175More user feedback will help to stabilize it.)
5176
5177@item Accepted Values:
5178@itemize
5179@item @code{all}.
5180For @acronym{LALR} and @acronym{IELR} parsers (@pxref{Decl
5181Summary,,lr.type}) by default, all states are permitted to contain
5182default reductions.
5183The advantage is that parser table sizes can be significantly reduced.
5184The reason Bison does not by default attempt to address the disadvantage
5185of delayed syntax error detection is that this disadvantage is already
5186inherent in @acronym{LALR} and @acronym{IELR} parser tables.
5187That is, unlike in a canonical @acronym{LR} state, the lookahead sets of
5188reductions in an @acronym{LALR} or @acronym{IELR} state can contain
5189tokens that are syntactically incorrect for some left contexts.
5190
5191@item @code{consistent}.
5192@cindex consistent states
5193A consistent state is a state that has only one possible action.
5194If that action is a reduction, then the parser does not need to request
5195a lookahead token from the scanner before performing that action.
5196However, the parser only recognizes the ability to ignore the lookahead
5197token when such a reduction is encoded as a default reduction.
5198Thus, if default reductions are permitted in and only in consistent
5199states, then a canonical @acronym{LR} parser reports a syntax error as
5200soon as it @emph{needs} the syntactically unacceptable token from the
5201scanner.
5202
5203@item @code{accepting}.
5204@cindex accepting state
5205By default, the only default reduction permitted in a canonical
5206@acronym{LR} parser is the accept action in the accepting state, which
5207the parser reaches only after reading all tokens from the input.
5208Thus, the default canonical @acronym{LR} parser reports a syntax error
5209as soon as it @emph{reaches} the syntactically unacceptable token
5210without performing any extra reductions.
5211@end itemize
5212
5213@item Default Value:
5214@itemize
5215@item @code{accepting} if @code{lr.type} is @code{canonical-lr}.
5216@item @code{all} otherwise.
5217@end itemize
5218@end itemize
5219
5220@c ============================================ lr.keep-unreachable-states
5221
5222@item lr.keep-unreachable-states
5223@findex %define lr.keep-unreachable-states
5224
5225@itemize @bullet
5226@item Language(s): all
5227
5228@item Purpose: Requests that Bison allow unreachable parser states to remain in
5229the parser tables.
5230Bison considers a state to be unreachable if there exists no sequence of
5231transitions from the start state to that state.
5232A state can become unreachable during conflict resolution if Bison disables a
5233shift action leading to it from a predecessor state.
5234Keeping unreachable states is sometimes useful for analysis purposes, but they
5235are useless in the generated parser.
5236
5237@item Accepted Values: Boolean
5238
5239@item Default Value: @code{false}
5240
5241@item Caveats:
5242
5243@itemize @bullet
5244
5245@item Unreachable states may contain conflicts and may use rules not used in
5246any other state.
5247Thus, keeping unreachable states may induce warnings that are irrelevant to
5248your parser's behavior, and it may eliminate warnings that are relevant.
5249Of course, the change in warnings may actually be relevant to a parser table
5250analysis that wants to keep unreachable states, so this behavior will likely
5251remain in future Bison releases.
5252
5253@item While Bison is able to remove unreachable states, it is not guaranteed to
5254remove other kinds of useless states.
5255Specifically, when Bison disables reduce actions during conflict resolution,
5256some goto actions may become useless, and thus some additional states may
5257become useless.
5258If Bison were to compute which goto actions were useless and then disable those
5259actions, it could identify such states as unreachable and then remove those
5260states.
5261However, Bison does not compute which goto actions are useless.
5262@end itemize
5263@end itemize
5264@c lr.keep-unreachable-states
5265
5266@c ================================================== lr.type
5267
5268@item lr.type
5269@findex %define lr.type
5270@cindex @acronym{LALR}
5271@cindex @acronym{IELR}
5272@cindex @acronym{LR}
5273
5274@itemize @bullet
5275@item Language(s): all
5276
5277@item Purpose: Specifies the type of parser tables within the
5278@acronym{LR}(1) family.
5279(This feature is experimental.
5280More user feedback will help to stabilize it.)
5281
5282@item Accepted Values:
5283@itemize
5284@item @code{lalr}.
5285While Bison generates @acronym{LALR} parser tables by default for
5286historical reasons, @acronym{IELR} or canonical @acronym{LR} is almost
5287always preferable for deterministic parsers.
5288The trouble is that @acronym{LALR} parser tables can suffer from
5289mysterious conflicts and thus may not accept the full set of sentences
5290that @acronym{IELR} and canonical @acronym{LR} accept.
5291@xref{Mystery Conflicts}, for details.
5292However, there are at least two scenarios where @acronym{LALR} may be
5293worthwhile:
5294@itemize
5295@cindex @acronym{GLR} with @acronym{LALR}
5296@item When employing @acronym{GLR} parsers (@pxref{GLR Parsers}), if you
5297do not resolve any conflicts statically (for example, with @code{%left}
5298or @code{%prec}), then the parser explores all potential parses of any
5299given input.
5300In this case, the use of @acronym{LALR} parser tables is guaranteed not
5301to alter the language accepted by the parser.
5302@acronym{LALR} parser tables are the smallest parser tables Bison can
5303currently generate, so they may be preferable.
5304
5305@item Occasionally during development, an especially malformed grammar
5306with a major recurring flaw may severely impede the @acronym{IELR} or
5307canonical @acronym{LR} parser table generation algorithm.
5308@acronym{LALR} can be a quick way to generate parser tables in order to
5309investigate such problems while ignoring the more subtle differences
5310from @acronym{IELR} and canonical @acronym{LR}.
5311@end itemize
5312
5313@item @code{ielr}.
5314@acronym{IELR} is a minimal @acronym{LR} algorithm.
5315That is, given any grammar (@acronym{LR} or non-@acronym{LR}),
5316@acronym{IELR} and canonical @acronym{LR} always accept exactly the same
5317set of sentences.
5318However, as for @acronym{LALR}, the number of parser states is often an
5319order of magnitude less for @acronym{IELR} than for canonical
5320@acronym{LR}.
5321More importantly, because canonical @acronym{LR}'s extra parser states
5322may contain duplicate conflicts in the case of non-@acronym{LR}
5323grammars, the number of conflicts for @acronym{IELR} is often an order
5324of magnitude less as well.
5325This can significantly reduce the complexity of developing of a grammar.
5326
5327@item @code{canonical-lr}.
5328@cindex delayed syntax errors
5329@cindex syntax errors delayed
5330The only advantage of canonical @acronym{LR} over @acronym{IELR} is
5331that, for every left context of every canonical @acronym{LR} state, the
5332set of tokens accepted by that state is the exact set of tokens that is
5333syntactically acceptable in that left context.
5334Thus, the only difference in parsing behavior is that the canonical
5335@acronym{LR} parser can report a syntax error as soon as possible
5336without performing any unnecessary reductions.
5337@xref{Decl Summary,,lr.default-reductions}, for further details.
5338Even when canonical @acronym{LR} behavior is ultimately desired,
5339@acronym{IELR}'s elimination of duplicate conflicts should still
5340facilitate the development of a grammar.
5341@end itemize
5342
5343@item Default Value: @code{lalr}
5344@end itemize
5345
5346
5347@c ================================================== namespace
5348@item namespace
5349@findex %define namespace
5350Obsoleted by @code{api.namespace}
5351@c namespace
5352
5353
5354@c ================================================== parse.assert
5355@item parse.assert
5356@findex %define parse.assert
5357
5358@itemize
5359@item Languages(s): C++
5360
5361@item Purpose: Issue runtime assertions to catch invalid uses.
5362In C++, when variants are used (@pxref{C++ Variants}), symbols must be
5363constructed and
5364destroyed properly. This option checks these constraints.
5365
5366@item Accepted Values: Boolean
5367
5368@item Default Value: @code{false}
5369@end itemize
5370@c parse.assert
5371
5372
5373@c ================================================== parse.error
5374@item parse.error
5375@findex %define parse.error
5376@itemize
5377@item Languages(s):
5378all.
5379@item Purpose:
5380Control the kind of error messages passed to the error reporting
5381function. @xref{Error Reporting, ,The Error Reporting Function
5382@code{yyerror}}.
5383@item Accepted Values:
5384@itemize
5385@item @code{simple}
5386Error messages passed to @code{yyerror} are simply @w{@code{"syntax
5387error"}}.
5388@item @code{verbose}
5389Error messages report the unexpected token, and possibly the expected
5390ones.
5391@end itemize
5392
5393@item Default Value:
5394@code{simple}
5395@end itemize
5396@c parse.error
5397
5398
5399@c ================================================== parse.trace
5400@item parse.trace
5401@findex %define parse.trace
5402
5403@itemize
5404@item Languages(s): C, C++
5405
5406@item Purpose: Require parser instrumentation for tracing.
5407In C/C++, define the macro @code{YYDEBUG} to 1 in the parser file if it
5408is not already defined, so that the debugging facilities are compiled.
5409@xref{Tracing, ,Tracing Your Parser}.
5410
5411@item Accepted Values: Boolean
5412
5413@item Default Value: @code{false}
5414@end itemize
5415@c parse.trace
5416
5417@c ================================================== variant
5418@item variant
5419@findex %define variant
5420
5421@itemize @bullet
5422@item Language(s):
5423C++
5424
5425@item Purpose:
5426Requests variant-based semantic values.
5427@xref{C++ Variants}.
5428
5429@item Accepted Values:
5430Boolean.
5431
5432@item Default Value:
5433@code{false}
5434@end itemize
5435@c variant
5436
5437
5438@end table
5439@end deffn
5440@c ---------------------------------------------------------- %define
5441
5442@deffn {Directive} %defines
5443Write a header file containing macro definitions for the token type
5444names defined in the grammar as well as a few other declarations.
5445If the parser output file is named @file{@var{name}.c} then this file
5446is named @file{@var{name}.h}.
5447
5448For C parsers, the output header declares @code{YYSTYPE} unless
5449@code{YYSTYPE} is already defined as a macro or you have used a
5450@code{<@var{type}>} tag without using @code{%union}.
5451Therefore, if you are using a @code{%union}
5452(@pxref{Multiple Types, ,More Than One Value Type}) with components that
5453require other definitions, or if you have defined a @code{YYSTYPE} macro
5454or type definition
5455(@pxref{Value Type, ,Data Types of Semantic Values}), you need to
5456arrange for these definitions to be propagated to all modules, e.g., by
5457putting them in a prerequisite header that is included both by your
5458parser and by any other module that needs @code{YYSTYPE}.
5459
5460Unless your parser is pure, the output header declares @code{yylval}
5461as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
5462Parser}.
5463
5464If you have also used locations, the output header declares
5465@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
5466the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations, ,Tracking
5467Locations}.
5468
5469This output file is normally essential if you wish to put the definition
5470of @code{yylex} in a separate source file, because @code{yylex}
5471typically needs to be able to refer to the above-mentioned declarations
5472and to the token type codes. @xref{Token Values, ,Semantic Values of
5473Tokens}.
5474
5475@findex %code requires
5476@findex %code provides
5477If you have declared @code{%code requires} or @code{%code provides}, the output
5478header also contains their code.
5479@xref{Decl Summary, ,%code}.
5480@end deffn
5481
5482@deffn {Directive} %defines @var{defines-file}
5483Same as above, but save in the file @var{defines-file}.
5484@end deffn
5485
5486@deffn {Directive} %destructor
5487Specify how the parser should reclaim the memory associated to
5488discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
5489@end deffn
5490
5491@deffn {Directive} %file-prefix "@var{prefix}"
5492Specify a prefix to use for all Bison output file names. The names are
5493chosen as if the input file were named @file{@var{prefix}.y}.
5494@end deffn
5495
5496@deffn {Directive} %language "@var{language}"
5497Specify the programming language for the generated parser. Currently
5498supported languages include C, C++, and Java.
5499@var{language} is case-insensitive.
5500
5501This directive is experimental and its effect may be modified in future
5502releases.
5503@end deffn
5504
5505@deffn {Directive} %locations
5506Generate the code processing the locations (@pxref{Action Features,
5507,Special Features for Use in Actions}). This mode is enabled as soon as
5508the grammar uses the special @samp{@@@var{n}} tokens, but if your
5509grammar does not use it, using @samp{%locations} allows for more
5510accurate syntax error messages.
5511@end deffn
5512
5513@deffn {Directive} %name-prefix "@var{prefix}"
5514Rename the external symbols used in the parser so that they start with
5515@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
5516in C parsers
5517is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5518@code{yylval}, @code{yychar}, @code{yydebug}, and
5519(if locations are used) @code{yylloc}. If you use a push parser,
5520@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5521@code{yypstate_new} and @code{yypstate_delete} will
5522also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
5523names become @code{c_parse}, @code{c_lex}, and so on.
5524For C++ parsers, see the @samp{%define api.namespace} documentation in this
5525section.
5526@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
5527@end deffn
5528
5529@ifset defaultprec
5530@deffn {Directive} %no-default-prec
5531Do not assign a precedence to rules lacking an explicit @code{%prec}
5532modifier (@pxref{Contextual Precedence, ,Context-Dependent
5533Precedence}).
5534@end deffn
5535@end ifset
5536
5537@deffn {Directive} %no-lines
5538Don't generate any @code{#line} preprocessor commands in the parser
5539file. Ordinarily Bison writes these commands in the parser file so that
5540the C compiler and debuggers will associate errors and object code with
5541your source file (the grammar file). This directive causes them to
5542associate errors with the parser file, treating it an independent source
5543file in its own right.
5544@end deffn
5545
5546@deffn {Directive} %output "@var{file}"
5547Specify @var{file} for the parser file.
5548@end deffn
5549
5550@deffn {Directive} %pure-parser
5551Deprecated version of @samp{%define api.pure} (@pxref{Decl Summary, ,%define}),
5552for which Bison is more careful to warn about unreasonable usage.
5553@end deffn
5554
5555@deffn {Directive} %require "@var{version}"
5556Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5557Require a Version of Bison}.
5558@end deffn
5559
5560@deffn {Directive} %skeleton "@var{file}"
5561Specify the skeleton to use.
5562
5563@c You probably don't need this option unless you are developing Bison.
5564@c You should use @code{%language} if you want to specify the skeleton for a
5565@c different language, because it is clearer and because it will always choose the
5566@c correct skeleton for non-deterministic or push parsers.
5567
5568If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5569file in the Bison installation directory.
5570If it does, @var{file} is an absolute file name or a file name relative to the
5571directory of the grammar file.
5572This is similar to how most shells resolve commands.
5573@end deffn
5574
5575@deffn {Directive} %token-table
5576Generate an array of token names in the parser file. The name of the
5577array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
5578token whose internal Bison token code number is @var{i}. The first
5579three elements of @code{yytname} correspond to the predefined tokens
5580@code{"$end"},
5581@code{"error"}, and @code{"$undefined"}; after these come the symbols
5582defined in the grammar file.
5583
5584The name in the table includes all the characters needed to represent
5585the token in Bison. For single-character literals and literal
5586strings, this includes the surrounding quoting characters and any
5587escape sequences. For example, the Bison single-character literal
5588@code{'+'} corresponds to a three-character name, represented in C as
5589@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5590corresponds to a five-character name, represented in C as
5591@code{"\"\\\\/\""}.
5592
5593When you specify @code{%token-table}, Bison also generates macro
5594definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5595@code{YYNRULES}, and @code{YYNSTATES}:
5596
5597@table @code
5598@item YYNTOKENS
5599The highest token number, plus one.
5600@item YYNNTS
5601The number of nonterminal symbols.
5602@item YYNRULES
5603The number of grammar rules,
5604@item YYNSTATES
5605The number of parser states (@pxref{Parser States}).
5606@end table
5607@end deffn
5608
5609@deffn {Directive} %verbose
5610Write an extra output file containing verbose descriptions of the
5611parser states and what is done for each type of lookahead token in
5612that state. @xref{Understanding, , Understanding Your Parser}, for more
5613information.
5614@end deffn
5615
5616@deffn {Directive} %yacc
5617Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5618including its naming conventions. @xref{Bison Options}, for more.
5619@end deffn
5620
5621
5622@node Multiple Parsers
5623@section Multiple Parsers in the Same Program
5624
5625Most programs that use Bison parse only one language and therefore contain
5626only one Bison parser. But what if you want to parse more than one
5627language with the same program? Then you need to avoid a name conflict
5628between different definitions of @code{yyparse}, @code{yylval}, and so on.
5629
5630The easy way to do this is to use the option @samp{-p @var{prefix}}
5631(@pxref{Invocation, ,Invoking Bison}). This renames the interface
5632functions and variables of the Bison parser to start with @var{prefix}
5633instead of @samp{yy}. You can use this to give each parser distinct
5634names that do not conflict.
5635
5636The precise list of symbols renamed is @code{yyparse}, @code{yylex},
5637@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
5638@code{yychar} and @code{yydebug}. If you use a push parser,
5639@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5640@code{yypstate_new} and @code{yypstate_delete} will also be renamed.
5641For example, if you use @samp{-p c}, the names become @code{cparse},
5642@code{clex}, and so on.
5643
5644@strong{All the other variables and macros associated with Bison are not
5645renamed.} These others are not global; there is no conflict if the same
5646name is used in different parsers. For example, @code{YYSTYPE} is not
5647renamed, but defining this in different ways in different parsers causes
5648no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5649
5650The @samp{-p} option works by adding macro definitions to the beginning
5651of the parser source file, defining @code{yyparse} as
5652@code{@var{prefix}parse}, and so on. This effectively substitutes one
5653name for the other in the entire parser file.
5654
5655@node Interface
5656@chapter Parser C-Language Interface
5657@cindex C-language interface
5658@cindex interface
5659
5660The Bison parser is actually a C function named @code{yyparse}. Here we
5661describe the interface conventions of @code{yyparse} and the other
5662functions that it needs to use.
5663
5664Keep in mind that the parser uses many C identifiers starting with
5665@samp{yy} and @samp{YY} for internal purposes. If you use such an
5666identifier (aside from those in this manual) in an action or in epilogue
5667in the grammar file, you are likely to run into trouble.
5668
5669@menu
5670* Parser Function:: How to call @code{yyparse} and what it returns.
5671* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5672* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
5673* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
5674* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
5675* Lexical:: You must supply a function @code{yylex}
5676 which reads tokens.
5677* Error Reporting:: You must supply a function @code{yyerror}.
5678* Action Features:: Special features for use in actions.
5679* Internationalization:: How to let the parser speak in the user's
5680 native language.
5681@end menu
5682
5683@node Parser Function
5684@section The Parser Function @code{yyparse}
5685@findex yyparse
5686
5687You call the function @code{yyparse} to cause parsing to occur. This
5688function reads tokens, executes actions, and ultimately returns when it
5689encounters end-of-input or an unrecoverable syntax error. You can also
5690write an action which directs @code{yyparse} to return immediately
5691without reading further.
5692
5693
5694@deftypefun int yyparse (void)
5695The value returned by @code{yyparse} is 0 if parsing was successful (return
5696is due to end-of-input).
5697
5698The value is 1 if parsing failed because of invalid input, i.e., input
5699that contains a syntax error or that causes @code{YYABORT} to be
5700invoked.
5701
5702The value is 2 if parsing failed due to memory exhaustion.
5703@end deftypefun
5704
5705In an action, you can cause immediate return from @code{yyparse} by using
5706these macros:
5707
5708@defmac YYACCEPT
5709@findex YYACCEPT
5710Return immediately with value 0 (to report success).
5711@end defmac
5712
5713@defmac YYABORT
5714@findex YYABORT
5715Return immediately with value 1 (to report failure).
5716@end defmac
5717
5718If you use a reentrant parser, you can optionally pass additional
5719parameter information to it in a reentrant way. To do so, use the
5720declaration @code{%parse-param}:
5721
5722@deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
5723@findex %parse-param
5724Declare that one or more
5725@var{argument-declaration} are additional @code{yyparse} arguments.
5726The @var{argument-declaration} is used when declaring
5727functions or prototypes. The last identifier in
5728@var{argument-declaration} must be the argument name.
5729@end deffn
5730
5731Here's an example. Write this in the parser:
5732
5733@example
5734%parse-param @{int *nastiness@} @{int *randomness@}
5735@end example
5736
5737@noindent
5738Then call the parser like this:
5739
5740@example
5741@{
5742 int nastiness, randomness;
5743 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5744 value = yyparse (&nastiness, &randomness);
5745 @dots{}
5746@}
5747@end example
5748
5749@noindent
5750In the grammar actions, use expressions like this to refer to the data:
5751
5752@example
5753exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5754@end example
5755
5756@node Push Parser Function
5757@section The Push Parser Function @code{yypush_parse}
5758@findex yypush_parse
5759
5760(The current push parsing interface is experimental and may evolve.
5761More user feedback will help to stabilize it.)
5762
5763You call the function @code{yypush_parse} to parse a single token. This
5764function is available if either the @samp{%define api.push-pull push} or
5765@samp{%define api.push-pull both} declaration is used.
5766@xref{Push Decl, ,A Push Parser}.
5767
5768@deftypefun int yypush_parse (yypstate *yyps)
5769The value returned by @code{yypush_parse} is the same as for yyparse with the
5770following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5771is required to finish parsing the grammar.
5772@end deftypefun
5773
5774@node Pull Parser Function
5775@section The Pull Parser Function @code{yypull_parse}
5776@findex yypull_parse
5777
5778(The current push parsing interface is experimental and may evolve.
5779More user feedback will help to stabilize it.)
5780
5781You call the function @code{yypull_parse} to parse the rest of the input
5782stream. This function is available if the @samp{%define api.push-pull both}
5783declaration is used.
5784@xref{Push Decl, ,A Push Parser}.
5785
5786@deftypefun int yypull_parse (yypstate *yyps)
5787The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
5788@end deftypefun
5789
5790@node Parser Create Function
5791@section The Parser Create Function @code{yystate_new}
5792@findex yypstate_new
5793
5794(The current push parsing interface is experimental and may evolve.
5795More user feedback will help to stabilize it.)
5796
5797You call the function @code{yypstate_new} to create a new parser instance.
5798This function is available if either the @samp{%define api.push-pull push} or
5799@samp{%define api.push-pull both} declaration is used.
5800@xref{Push Decl, ,A Push Parser}.
5801
5802@deftypefun yypstate *yypstate_new (void)
5803The function will return a valid parser instance if there was memory available
5804or 0 if no memory was available.
5805In impure mode, it will also return 0 if a parser instance is currently
5806allocated.
5807@end deftypefun
5808
5809@node Parser Delete Function
5810@section The Parser Delete Function @code{yystate_delete}
5811@findex yypstate_delete
5812
5813(The current push parsing interface is experimental and may evolve.
5814More user feedback will help to stabilize it.)
5815
5816You call the function @code{yypstate_delete} to delete a parser instance.
5817function is available if either the @samp{%define api.push-pull push} or
5818@samp{%define api.push-pull both} declaration is used.
5819@xref{Push Decl, ,A Push Parser}.
5820
5821@deftypefun void yypstate_delete (yypstate *yyps)
5822This function will reclaim the memory associated with a parser instance.
5823After this call, you should no longer attempt to use the parser instance.
5824@end deftypefun
5825
5826@node Lexical
5827@section The Lexical Analyzer Function @code{yylex}
5828@findex yylex
5829@cindex lexical analyzer
5830
5831The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
5832the input stream and returns them to the parser. Bison does not create
5833this function automatically; you must write it so that @code{yyparse} can
5834call it. The function is sometimes referred to as a lexical scanner.
5835
5836In simple programs, @code{yylex} is often defined at the end of the Bison
5837grammar file. If @code{yylex} is defined in a separate source file, you
5838need to arrange for the token-type macro definitions to be available there.
5839To do this, use the @samp{-d} option when you run Bison, so that it will
5840write these macro definitions into a separate header file
5841@file{@var{name}.tab.h} which you can include in the other source files
5842that need it. @xref{Invocation, ,Invoking Bison}.
5843
5844@menu
5845* Calling Convention:: How @code{yyparse} calls @code{yylex}.
5846* Token Values:: How @code{yylex} must return the semantic value
5847 of the token it has read.
5848* Token Locations:: How @code{yylex} must return the text location
5849 (line number, etc.) of the token, if the
5850 actions want that.
5851* Pure Calling:: How the calling convention differs in a pure parser
5852 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
5853@end menu
5854
5855@node Calling Convention
5856@subsection Calling Convention for @code{yylex}
5857
5858The value that @code{yylex} returns must be the positive numeric code
5859for the type of token it has just found; a zero or negative value
5860signifies end-of-input.
5861
5862When a token is referred to in the grammar rules by a name, that name
5863in the parser file becomes a C macro whose definition is the proper
5864numeric code for that token type. So @code{yylex} can use the name
5865to indicate that type. @xref{Symbols}.
5866
5867When a token is referred to in the grammar rules by a character literal,
5868the numeric code for that character is also the code for the token type.
5869So @code{yylex} can simply return that character code, possibly converted
5870to @code{unsigned char} to avoid sign-extension. The null character
5871must not be used this way, because its code is zero and that
5872signifies end-of-input.
5873
5874Here is an example showing these things:
5875
5876@example
5877int
5878yylex (void)
5879@{
5880 @dots{}
5881 if (c == EOF) /* Detect end-of-input. */
5882 return 0;
5883 @dots{}
5884 if (c == '+' || c == '-')
5885 return c; /* Assume token type for `+' is '+'. */
5886 @dots{}
5887 return INT; /* Return the type of the token. */
5888 @dots{}
5889@}
5890@end example
5891
5892@noindent
5893This interface has been designed so that the output from the @code{lex}
5894utility can be used without change as the definition of @code{yylex}.
5895
5896If the grammar uses literal string tokens, there are two ways that
5897@code{yylex} can determine the token type codes for them:
5898
5899@itemize @bullet
5900@item
5901If the grammar defines symbolic token names as aliases for the
5902literal string tokens, @code{yylex} can use these symbolic names like
5903all others. In this case, the use of the literal string tokens in
5904the grammar file has no effect on @code{yylex}.
5905
5906@item
5907@code{yylex} can find the multicharacter token in the @code{yytname}
5908table. The index of the token in the table is the token type's code.
5909The name of a multicharacter token is recorded in @code{yytname} with a
5910double-quote, the token's characters, and another double-quote. The
5911token's characters are escaped as necessary to be suitable as input
5912to Bison.
5913
5914Here's code for looking up a multicharacter token in @code{yytname},
5915assuming that the characters of the token are stored in
5916@code{token_buffer}, and assuming that the token does not contain any
5917characters like @samp{"} that require escaping.
5918
5919@smallexample
5920for (i = 0; i < YYNTOKENS; i++)
5921 @{
5922 if (yytname[i] != 0
5923 && yytname[i][0] == '"'
5924 && ! strncmp (yytname[i] + 1, token_buffer,
5925 strlen (token_buffer))
5926 && yytname[i][strlen (token_buffer) + 1] == '"'
5927 && yytname[i][strlen (token_buffer) + 2] == 0)
5928 break;
5929 @}
5930@end smallexample
5931
5932The @code{yytname} table is generated only if you use the
5933@code{%token-table} declaration. @xref{Decl Summary}.
5934@end itemize
5935
5936@node Token Values
5937@subsection Semantic Values of Tokens
5938
5939@vindex yylval
5940In an ordinary (nonreentrant) parser, the semantic value of the token must
5941be stored into the global variable @code{yylval}. When you are using
5942just one data type for semantic values, @code{yylval} has that type.
5943Thus, if the type is @code{int} (the default), you might write this in
5944@code{yylex}:
5945
5946@example
5947@group
5948 @dots{}
5949 yylval = value; /* Put value onto Bison stack. */
5950 return INT; /* Return the type of the token. */
5951 @dots{}
5952@end group
5953@end example
5954
5955When you are using multiple data types, @code{yylval}'s type is a union
5956made from the @code{%union} declaration (@pxref{Union Decl, ,The
5957Collection of Value Types}). So when you store a token's value, you
5958must use the proper member of the union. If the @code{%union}
5959declaration looks like this:
5960
5961@example
5962@group
5963%union @{
5964 int intval;
5965 double val;
5966 symrec *tptr;
5967@}
5968@end group
5969@end example
5970
5971@noindent
5972then the code in @code{yylex} might look like this:
5973
5974@example
5975@group
5976 @dots{}
5977 yylval.intval = value; /* Put value onto Bison stack. */
5978 return INT; /* Return the type of the token. */
5979 @dots{}
5980@end group
5981@end example
5982
5983@node Token Locations
5984@subsection Textual Locations of Tokens
5985
5986@vindex yylloc
5987If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
5988Tracking Locations}) in actions to keep track of the textual locations
5989of tokens and groupings, then you must provide this information in
5990@code{yylex}. The function @code{yyparse} expects to find the textual
5991location of a token just parsed in the global variable @code{yylloc}.
5992So @code{yylex} must store the proper data in that variable.
5993
5994By default, the value of @code{yylloc} is a structure and you need only
5995initialize the members that are going to be used by the actions. The
5996four members are called @code{first_line}, @code{first_column},
5997@code{last_line} and @code{last_column}. Note that the use of this
5998feature makes the parser noticeably slower.
5999
6000@tindex YYLTYPE
6001The data type of @code{yylloc} has the name @code{YYLTYPE}.
6002
6003@node Pure Calling
6004@subsection Calling Conventions for Pure Parsers
6005
6006When you use the Bison declaration @samp{%define api.pure} to request a
6007pure, reentrant parser, the global communication variables @code{yylval}
6008and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
6009Parser}.) In such parsers the two global variables are replaced by
6010pointers passed as arguments to @code{yylex}. You must declare them as
6011shown here, and pass the information back by storing it through those
6012pointers.
6013
6014@example
6015int
6016yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
6017@{
6018 @dots{}
6019 *lvalp = value; /* Put value onto Bison stack. */
6020 return INT; /* Return the type of the token. */
6021 @dots{}
6022@}
6023@end example
6024
6025If the grammar file does not use the @samp{@@} constructs to refer to
6026textual locations, then the type @code{YYLTYPE} will not be defined. In
6027this case, omit the second argument; @code{yylex} will be called with
6028only one argument.
6029
6030If you wish to pass additional arguments to @code{yylex}, use
6031@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
6032Function}). To pass additional arguments to both @code{yylex} and
6033@code{yyparse}, use @code{%param}.
6034
6035@deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
6036@findex %lex-param
6037Specify that @var{argument-declaration} are additional @code{yylex} argument
6038declarations. You may pass one or more such declarations, which is
6039equivalent to repeating @code{%lex-param}.
6040@end deffn
6041
6042@deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
6043@findex %param
6044Specify that @var{argument-declaration} are additional
6045@code{yylex}/@code{yyparse} argument declaration. This is equivalent to
6046@samp{%lex-param @{@var{argument-declaration}@} @dots{} %parse-param
6047@{@var{argument-declaration}@} @dots{}}. You may pass one or more
6048declarations, which is equivalent to repeating @code{%param}.
6049@end deffn
6050
6051For instance:
6052
6053@example
6054%lex-param @{scanner_mode *mode@}
6055%parse-param @{parser_mode *mode@}
6056%param @{environment_type *env@}
6057@end example
6058
6059@noindent
6060results in the following signature:
6061
6062@example
6063int yylex (scanner_mode *mode, environment_type *env);
6064int yyparse (parser_mode *mode, environment_type *env);
6065@end example
6066
6067If @samp{%define api.pure} is added:
6068
6069@example
6070int yylex (YYSTYPE *lvalp, scanner_mode *mode, environment_type *env);
6071int yyparse (parser_mode *mode, environment_type *env);
6072@end example
6073
6074@noindent
6075and finally, if both @samp{%define api.pure} and @code{%locations} are used:
6076
6077@example
6078int yylex (YYSTYPE *lvalp, YYLTYPE *llocp,
6079 scanner_mode *mode, environment_type *env);
6080int yyparse (parser_mode *mode, environment_type *env);
6081@end example
6082
6083@node Error Reporting
6084@section The Error Reporting Function @code{yyerror}
6085@cindex error reporting function
6086@findex yyerror
6087@cindex parse error
6088@cindex syntax error
6089
6090The Bison parser detects a @dfn{syntax error} (or @dfn{parse error})
6091whenever it reads a token which cannot satisfy any syntax rule. An
6092action in the grammar can also explicitly proclaim an error, using the
6093macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
6094in Actions}).
6095
6096The Bison parser expects to report the error by calling an error
6097reporting function named @code{yyerror}, which you must supply. It is
6098called by @code{yyparse} whenever a syntax error is found, and it
6099receives one argument. For a syntax error, the string is normally
6100@w{@code{"syntax error"}}.
6101
6102@findex %define parse.error
6103If you invoke @samp{%define parse.error verbose} in the Bison
6104declarations section (@pxref{Bison Declarations, ,The Bison Declarations
6105Section}), then Bison provides a more verbose and specific error message
6106string instead of just plain @w{@code{"syntax error"}}.
6107
6108The parser can detect one other kind of error: memory exhaustion. This
6109can happen when the input contains constructions that are very deeply
6110nested. It isn't likely you will encounter this, since the Bison
6111parser normally extends its stack automatically up to a very large limit. But
6112if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
6113fashion, except that the argument string is @w{@code{"memory exhausted"}}.
6114
6115In some cases diagnostics like @w{@code{"syntax error"}} are
6116translated automatically from English to some other language before
6117they are passed to @code{yyerror}. @xref{Internationalization}.
6118
6119The following definition suffices in simple programs:
6120
6121@example
6122@group
6123void
6124yyerror (char const *s)
6125@{
6126@end group
6127@group
6128 fprintf (stderr, "%s\n", s);
6129@}
6130@end group
6131@end example
6132
6133After @code{yyerror} returns to @code{yyparse}, the latter will attempt
6134error recovery if you have written suitable error recovery grammar rules
6135(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
6136immediately return 1.
6137
6138Obviously, in location tracking pure parsers, @code{yyerror} should have
6139an access to the current location.
6140This is indeed the case for the @acronym{GLR}
6141parsers, but not for the Yacc parser, for historical reasons. I.e., if
6142@samp{%locations %define api.pure} is passed then the prototypes for
6143@code{yyerror} are:
6144
6145@example
6146void yyerror (char const *msg); /* Yacc parsers. */
6147void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
6148@end example
6149
6150If @samp{%parse-param @{int *nastiness@}} is used, then:
6151
6152@example
6153void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
6154void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
6155@end example
6156
6157Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
6158convention for absolutely pure parsers, i.e., when the calling
6159convention of @code{yylex} @emph{and} the calling convention of
6160@samp{%define api.pure} are pure.
6161I.e.:
6162
6163@example
6164/* Location tracking. */
6165%locations
6166/* Pure yylex. */
6167%define api.pure
6168%lex-param @{int *nastiness@}
6169/* Pure yyparse. */
6170%parse-param @{int *nastiness@}
6171%parse-param @{int *randomness@}
6172@end example
6173
6174@noindent
6175results in the following signatures for all the parser kinds:
6176
6177@example
6178int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
6179int yyparse (int *nastiness, int *randomness);
6180void yyerror (YYLTYPE *locp,
6181 int *nastiness, int *randomness,
6182 char const *msg);
6183@end example
6184
6185@noindent
6186The prototypes are only indications of how the code produced by Bison
6187uses @code{yyerror}. Bison-generated code always ignores the returned
6188value, so @code{yyerror} can return any type, including @code{void}.
6189Also, @code{yyerror} can be a variadic function; that is why the
6190message is always passed last.
6191
6192Traditionally @code{yyerror} returns an @code{int} that is always
6193ignored, but this is purely for historical reasons, and @code{void} is
6194preferable since it more accurately describes the return type for
6195@code{yyerror}.
6196
6197@vindex yynerrs
6198The variable @code{yynerrs} contains the number of syntax errors
6199reported so far. Normally this variable is global; but if you
6200request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
6201then it is a local variable which only the actions can access.
6202
6203@node Action Features
6204@section Special Features for Use in Actions
6205@cindex summary, action features
6206@cindex action features summary
6207
6208Here is a table of Bison constructs, variables and macros that
6209are useful in actions.
6210
6211@deffn {Variable} $$
6212Acts like a variable that contains the semantic value for the
6213grouping made by the current rule. @xref{Actions}.
6214@end deffn
6215
6216@deffn {Variable} $@var{n}
6217Acts like a variable that contains the semantic value for the
6218@var{n}th component of the current rule. @xref{Actions}.
6219@end deffn
6220
6221@deffn {Variable} $<@var{typealt}>$
6222Like @code{$$} but specifies alternative @var{typealt} in the union
6223specified by the @code{%union} declaration. @xref{Action Types, ,Data
6224Types of Values in Actions}.
6225@end deffn
6226
6227@deffn {Variable} $<@var{typealt}>@var{n}
6228Like @code{$@var{n}} but specifies alternative @var{typealt} in the
6229union specified by the @code{%union} declaration.
6230@xref{Action Types, ,Data Types of Values in Actions}.
6231@end deffn
6232
6233@deffn {Macro} YYABORT;
6234Return immediately from @code{yyparse}, indicating failure.
6235@xref{Parser Function, ,The Parser Function @code{yyparse}}.
6236@end deffn
6237
6238@deffn {Macro} YYACCEPT;
6239Return immediately from @code{yyparse}, indicating success.
6240@xref{Parser Function, ,The Parser Function @code{yyparse}}.
6241@end deffn
6242
6243@deffn {Macro} YYBACKUP (@var{token}, @var{value});
6244@findex YYBACKUP
6245Unshift a token. This macro is allowed only for rules that reduce
6246a single value, and only when there is no lookahead token.
6247It is also disallowed in @acronym{GLR} parsers.
6248It installs a lookahead token with token type @var{token} and
6249semantic value @var{value}; then it discards the value that was
6250going to be reduced by this rule.
6251
6252If the macro is used when it is not valid, such as when there is
6253a lookahead token already, then it reports a syntax error with
6254a message @samp{cannot back up} and performs ordinary error
6255recovery.
6256
6257In either case, the rest of the action is not executed.
6258@end deffn
6259
6260@deffn {Macro} YYEMPTY
6261@vindex YYEMPTY
6262Value stored in @code{yychar} when there is no lookahead token.
6263@end deffn
6264
6265@deffn {Macro} YYEOF
6266@vindex YYEOF
6267Value stored in @code{yychar} when the lookahead is the end of the input
6268stream.
6269@end deffn
6270
6271@deffn {Macro} YYERROR;
6272@findex YYERROR
6273Cause an immediate syntax error. This statement initiates error
6274recovery just as if the parser itself had detected an error; however, it
6275does not call @code{yyerror}, and does not print any message. If you
6276want to print an error message, call @code{yyerror} explicitly before
6277the @samp{YYERROR;} statement. @xref{Error Recovery}.
6278@end deffn
6279
6280@deffn {Macro} YYRECOVERING
6281@findex YYRECOVERING
6282The expression @code{YYRECOVERING ()} yields 1 when the parser
6283is recovering from a syntax error, and 0 otherwise.
6284@xref{Error Recovery}.
6285@end deffn
6286
6287@deffn {Variable} yychar
6288Variable containing either the lookahead token, or @code{YYEOF} when the
6289lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
6290has been performed so the next token is not yet known.
6291Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
6292Actions}).
6293@xref{Lookahead, ,Lookahead Tokens}.
6294@end deffn
6295
6296@deffn {Macro} yyclearin;
6297Discard the current lookahead token. This is useful primarily in
6298error rules.
6299Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
6300Semantic Actions}).
6301@xref{Error Recovery}.
6302@end deffn
6303
6304@deffn {Macro} yyerrok;
6305Resume generating error messages immediately for subsequent syntax
6306errors. This is useful primarily in error rules.
6307@xref{Error Recovery}.
6308@end deffn
6309
6310@deffn {Variable} yylloc
6311Variable containing the lookahead token location when @code{yychar} is not set
6312to @code{YYEMPTY} or @code{YYEOF}.
6313Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
6314Actions}).
6315@xref{Actions and Locations, ,Actions and Locations}.
6316@end deffn
6317
6318@deffn {Variable} yylval
6319Variable containing the lookahead token semantic value when @code{yychar} is
6320not set to @code{YYEMPTY} or @code{YYEOF}.
6321Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
6322Actions}).
6323@xref{Actions, ,Actions}.
6324@end deffn
6325
6326@deffn {Value} @@$
6327@findex @@$
6328Acts like a structure variable containing information on the textual location
6329of the grouping made by the current rule. @xref{Locations, ,
6330Tracking Locations}.
6331
6332@c Check if those paragraphs are still useful or not.
6333
6334@c @example
6335@c struct @{
6336@c int first_line, last_line;
6337@c int first_column, last_column;
6338@c @};
6339@c @end example
6340
6341@c Thus, to get the starting line number of the third component, you would
6342@c use @samp{@@3.first_line}.
6343
6344@c In order for the members of this structure to contain valid information,
6345@c you must make @code{yylex} supply this information about each token.
6346@c If you need only certain members, then @code{yylex} need only fill in
6347@c those members.
6348
6349@c The use of this feature makes the parser noticeably slower.
6350@end deffn
6351
6352@deffn {Value} @@@var{n}
6353@findex @@@var{n}
6354Acts like a structure variable containing information on the textual location
6355of the @var{n}th component of the current rule. @xref{Locations, ,
6356Tracking Locations}.
6357@end deffn
6358
6359@node Internationalization
6360@section Parser Internationalization
6361@cindex internationalization
6362@cindex i18n
6363@cindex NLS
6364@cindex gettext
6365@cindex bison-po
6366
6367A Bison-generated parser can print diagnostics, including error and
6368tracing messages. By default, they appear in English. However, Bison
6369also supports outputting diagnostics in the user's native language. To
6370make this work, the user should set the usual environment variables.
6371@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
6372For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
6373set the user's locale to French Canadian using the @acronym{UTF}-8
6374encoding. The exact set of available locales depends on the user's
6375installation.
6376
6377The maintainer of a package that uses a Bison-generated parser enables
6378the internationalization of the parser's output through the following
6379steps. Here we assume a package that uses @acronym{GNU} Autoconf and
6380@acronym{GNU} Automake.
6381
6382@enumerate
6383@item
6384@cindex bison-i18n.m4
6385Into the directory containing the @acronym{GNU} Autoconf macros used
6386by the package---often called @file{m4}---copy the
6387@file{bison-i18n.m4} file installed by Bison under
6388@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
6389For example:
6390
6391@example
6392cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
6393@end example
6394
6395@item
6396@findex BISON_I18N
6397@vindex BISON_LOCALEDIR
6398@vindex YYENABLE_NLS
6399In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
6400invocation, add an invocation of @code{BISON_I18N}. This macro is
6401defined in the file @file{bison-i18n.m4} that you copied earlier. It
6402causes @samp{configure} to find the value of the
6403@code{BISON_LOCALEDIR} variable, and it defines the source-language
6404symbol @code{YYENABLE_NLS} to enable translations in the
6405Bison-generated parser.
6406
6407@item
6408In the @code{main} function of your program, designate the directory
6409containing Bison's runtime message catalog, through a call to
6410@samp{bindtextdomain} with domain name @samp{bison-runtime}.
6411For example:
6412
6413@example
6414bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
6415@end example
6416
6417Typically this appears after any other call @code{bindtextdomain
6418(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
6419@samp{BISON_LOCALEDIR} to be defined as a string through the
6420@file{Makefile}.
6421
6422@item
6423In the @file{Makefile.am} that controls the compilation of the @code{main}
6424function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
6425either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
6426
6427@example
6428DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6429@end example
6430
6431or:
6432
6433@example
6434AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6435@end example
6436
6437@item
6438Finally, invoke the command @command{autoreconf} to generate the build
6439infrastructure.
6440@end enumerate
6441
6442
6443@node Algorithm
6444@chapter The Bison Parser Algorithm
6445@cindex Bison parser algorithm
6446@cindex algorithm of parser
6447@cindex shifting
6448@cindex reduction
6449@cindex parser stack
6450@cindex stack, parser
6451
6452As Bison reads tokens, it pushes them onto a stack along with their
6453semantic values. The stack is called the @dfn{parser stack}. Pushing a
6454token is traditionally called @dfn{shifting}.
6455
6456For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
6457@samp{3} to come. The stack will have four elements, one for each token
6458that was shifted.
6459
6460But the stack does not always have an element for each token read. When
6461the last @var{n} tokens and groupings shifted match the components of a
6462grammar rule, they can be combined according to that rule. This is called
6463@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
6464single grouping whose symbol is the result (left hand side) of that rule.
6465Running the rule's action is part of the process of reduction, because this
6466is what computes the semantic value of the resulting grouping.
6467
6468For example, if the infix calculator's parser stack contains this:
6469
6470@example
64711 + 5 * 3
6472@end example
6473
6474@noindent
6475and the next input token is a newline character, then the last three
6476elements can be reduced to 15 via the rule:
6477
6478@example
6479expr: expr '*' expr;
6480@end example
6481
6482@noindent
6483Then the stack contains just these three elements:
6484
6485@example
64861 + 15
6487@end example
6488
6489@noindent
6490At this point, another reduction can be made, resulting in the single value
649116. Then the newline token can be shifted.
6492
6493The parser tries, by shifts and reductions, to reduce the entire input down
6494to a single grouping whose symbol is the grammar's start-symbol
6495(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6496
6497This kind of parser is known in the literature as a bottom-up parser.
6498
6499@menu
6500* Lookahead:: Parser looks one token ahead when deciding what to do.
6501* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6502* Precedence:: Operator precedence works by resolving conflicts.
6503* Contextual Precedence:: When an operator's precedence depends on context.
6504* Parser States:: The parser is a finite-state-machine with stack.
6505* Reduce/Reduce:: When two rules are applicable in the same situation.
6506* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
6507* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
6508* Memory Management:: What happens when memory is exhausted. How to avoid it.
6509@end menu
6510
6511@node Lookahead
6512@section Lookahead Tokens
6513@cindex lookahead token
6514
6515The Bison parser does @emph{not} always reduce immediately as soon as the
6516last @var{n} tokens and groupings match a rule. This is because such a
6517simple strategy is inadequate to handle most languages. Instead, when a
6518reduction is possible, the parser sometimes ``looks ahead'' at the next
6519token in order to decide what to do.
6520
6521When a token is read, it is not immediately shifted; first it becomes the
6522@dfn{lookahead token}, which is not on the stack. Now the parser can
6523perform one or more reductions of tokens and groupings on the stack, while
6524the lookahead token remains off to the side. When no more reductions
6525should take place, the lookahead token is shifted onto the stack. This
6526does not mean that all possible reductions have been done; depending on the
6527token type of the lookahead token, some rules may choose to delay their
6528application.
6529
6530Here is a simple case where lookahead is needed. These three rules define
6531expressions which contain binary addition operators and postfix unary
6532factorial operators (@samp{!}), and allow parentheses for grouping.
6533
6534@example
6535@group
6536expr: term '+' expr
6537 | term
6538 ;
6539@end group
6540
6541@group
6542term: '(' expr ')'
6543 | term '!'
6544 | NUMBER
6545 ;
6546@end group
6547@end example
6548
6549Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6550should be done? If the following token is @samp{)}, then the first three
6551tokens must be reduced to form an @code{expr}. This is the only valid
6552course, because shifting the @samp{)} would produce a sequence of symbols
6553@w{@code{term ')'}}, and no rule allows this.
6554
6555If the following token is @samp{!}, then it must be shifted immediately so
6556that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6557parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6558@code{expr}. It would then be impossible to shift the @samp{!} because
6559doing so would produce on the stack the sequence of symbols @code{expr
6560'!'}. No rule allows that sequence.
6561
6562@vindex yychar
6563@vindex yylval
6564@vindex yylloc
6565The lookahead token is stored in the variable @code{yychar}.
6566Its semantic value and location, if any, are stored in the variables
6567@code{yylval} and @code{yylloc}.
6568@xref{Action Features, ,Special Features for Use in Actions}.
6569
6570@node Shift/Reduce
6571@section Shift/Reduce Conflicts
6572@cindex conflicts
6573@cindex shift/reduce conflicts
6574@cindex dangling @code{else}
6575@cindex @code{else}, dangling
6576
6577Suppose we are parsing a language which has if-then and if-then-else
6578statements, with a pair of rules like this:
6579
6580@example
6581@group
6582if_stmt:
6583 IF expr THEN stmt
6584 | IF expr THEN stmt ELSE stmt
6585 ;
6586@end group
6587@end example
6588
6589@noindent
6590Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6591terminal symbols for specific keyword tokens.
6592
6593When the @code{ELSE} token is read and becomes the lookahead token, the
6594contents of the stack (assuming the input is valid) are just right for
6595reduction by the first rule. But it is also legitimate to shift the
6596@code{ELSE}, because that would lead to eventual reduction by the second
6597rule.
6598
6599This situation, where either a shift or a reduction would be valid, is
6600called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6601these conflicts by choosing to shift, unless otherwise directed by
6602operator precedence declarations. To see the reason for this, let's
6603contrast it with the other alternative.
6604
6605Since the parser prefers to shift the @code{ELSE}, the result is to attach
6606the else-clause to the innermost if-statement, making these two inputs
6607equivalent:
6608
6609@example
6610if x then if y then win (); else lose;
6611
6612if x then do; if y then win (); else lose; end;
6613@end example
6614
6615But if the parser chose to reduce when possible rather than shift, the
6616result would be to attach the else-clause to the outermost if-statement,
6617making these two inputs equivalent:
6618
6619@example
6620if x then if y then win (); else lose;
6621
6622if x then do; if y then win (); end; else lose;
6623@end example
6624
6625The conflict exists because the grammar as written is ambiguous: either
6626parsing of the simple nested if-statement is legitimate. The established
6627convention is that these ambiguities are resolved by attaching the
6628else-clause to the innermost if-statement; this is what Bison accomplishes
6629by choosing to shift rather than reduce. (It would ideally be cleaner to
6630write an unambiguous grammar, but that is very hard to do in this case.)
6631This particular ambiguity was first encountered in the specifications of
6632Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6633
6634To avoid warnings from Bison about predictable, legitimate shift/reduce
6635conflicts, use the @code{%expect @var{n}} declaration. There will be no
6636warning as long as the number of shift/reduce conflicts is exactly @var{n}.
6637@xref{Expect Decl, ,Suppressing Conflict Warnings}.
6638
6639The definition of @code{if_stmt} above is solely to blame for the
6640conflict, but the conflict does not actually appear without additional
6641rules. Here is a complete Bison input file that actually manifests the
6642conflict:
6643
6644@example
6645@group
6646%token IF THEN ELSE variable
6647%%
6648@end group
6649@group
6650stmt: expr
6651 | if_stmt
6652 ;
6653@end group
6654
6655@group
6656if_stmt:
6657 IF expr THEN stmt
6658 | IF expr THEN stmt ELSE stmt
6659 ;
6660@end group
6661
6662expr: variable
6663 ;
6664@end example
6665
6666@node Precedence
6667@section Operator Precedence
6668@cindex operator precedence
6669@cindex precedence of operators
6670
6671Another situation where shift/reduce conflicts appear is in arithmetic
6672expressions. Here shifting is not always the preferred resolution; the
6673Bison declarations for operator precedence allow you to specify when to
6674shift and when to reduce.
6675
6676@menu
6677* Why Precedence:: An example showing why precedence is needed.
6678* Using Precedence:: How to specify precedence and associativity.
6679* Precedence Only:: How to specify precedence only.
6680* Precedence Examples:: How these features are used in the previous example.
6681* How Precedence:: How they work.
6682@end menu
6683
6684@node Why Precedence
6685@subsection When Precedence is Needed
6686
6687Consider the following ambiguous grammar fragment (ambiguous because the
6688input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6689
6690@example
6691@group
6692expr: expr '-' expr
6693 | expr '*' expr
6694 | expr '<' expr
6695 | '(' expr ')'
6696 @dots{}
6697 ;
6698@end group
6699@end example
6700
6701@noindent
6702Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
6703should it reduce them via the rule for the subtraction operator? It
6704depends on the next token. Of course, if the next token is @samp{)}, we
6705must reduce; shifting is invalid because no single rule can reduce the
6706token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6707the next token is @samp{*} or @samp{<}, we have a choice: either
6708shifting or reduction would allow the parse to complete, but with
6709different results.
6710
6711To decide which one Bison should do, we must consider the results. If
6712the next operator token @var{op} is shifted, then it must be reduced
6713first in order to permit another opportunity to reduce the difference.
6714The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6715hand, if the subtraction is reduced before shifting @var{op}, the result
6716is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6717reduce should depend on the relative precedence of the operators
6718@samp{-} and @var{op}: @samp{*} should be shifted first, but not
6719@samp{<}.
6720
6721@cindex associativity
6722What about input such as @w{@samp{1 - 2 - 5}}; should this be
6723@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6724operators we prefer the former, which is called @dfn{left association}.
6725The latter alternative, @dfn{right association}, is desirable for
6726assignment operators. The choice of left or right association is a
6727matter of whether the parser chooses to shift or reduce when the stack
6728contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
6729makes right-associativity.
6730
6731@node Using Precedence
6732@subsection Specifying Operator Precedence
6733@findex %left
6734@findex %nonassoc
6735@findex %precedence
6736@findex %right
6737
6738Bison allows you to specify these choices with the operator precedence
6739declarations @code{%left} and @code{%right}. Each such declaration
6740contains a list of tokens, which are operators whose precedence and
6741associativity is being declared. The @code{%left} declaration makes all
6742those operators left-associative and the @code{%right} declaration makes
6743them right-associative. A third alternative is @code{%nonassoc}, which
6744declares that it is a syntax error to find the same operator twice ``in a
6745row''.
6746The last alternative, @code{%precedence}, allows to define only
6747precedence and no associativity at all. As a result, any
6748associativity-related conflict that remains will be reported as an
6749compile-time error. The directive @code{%nonassoc} creates run-time
6750error: using the operator in a associative way is a syntax error. The
6751directive @code{%precedence} creates compile-time errors: an operator
6752@emph{can} be involved in an associativity-related conflict, contrary to
6753what expected the grammar author.
6754
6755The relative precedence of different operators is controlled by the
6756order in which they are declared. The first precedence/associativity
6757declaration in the file declares the operators whose
6758precedence is lowest, the next such declaration declares the operators
6759whose precedence is a little higher, and so on.
6760
6761@node Precedence Only
6762@subsection Specifying Precedence Only
6763@findex %precedence
6764
6765Since @acronym{POSIX} Yacc defines only @code{%left}, @code{%right}, and
6766@code{%nonassoc}, which all defines precedence and associativity, little
6767attention is paid to the fact that precedence cannot be defined without
6768defining associativity. Yet, sometimes, when trying to solve a
6769conflict, precedence suffices. In such a case, using @code{%left},
6770@code{%right}, or @code{%nonassoc} might hide future (associativity
6771related) conflicts that would remain hidden.
6772
6773The dangling @code{else} ambiguity (@pxref{Shift/Reduce, , Shift/Reduce
6774Conflicts}) can be solved explicitly. This shift/reduce conflicts occurs
6775in the following situation, where the period denotes the current parsing
6776state:
6777
6778@example
6779if @var{e1} then if @var{e2} then @var{s1} . else @var{s2}
6780@end example
6781
6782The conflict involves the reduction of the rule @samp{IF expr THEN
6783stmt}, which precedence is by default that of its last token
6784(@code{THEN}), and the shifting of the token @code{ELSE}. The usual
6785disambiguation (attach the @code{else} to the closest @code{if}),
6786shifting must be preferred, i.e., the precedence of @code{ELSE} must be
6787higher than that of @code{THEN}. But neither is expected to be involved
6788in an associativity related conflict, which can be specified as follows.
6789
6790@example
6791%precedence THEN
6792%precedence ELSE
6793@end example
6794
6795The unary-minus is another typical example where associativity is
6796usually over-specified, see @ref{Infix Calc, , Infix Notation
6797Calculator: @code{calc}}. The @code{%left} directive is traditionally
6798used to declare the precedence of @code{NEG}, which is more than needed
6799since it also defines its associativity. While this is harmless in the
6800traditional example, who knows how @code{NEG} might be used in future
6801evolutions of the grammar@dots{}
6802
6803@node Precedence Examples
6804@subsection Precedence Examples
6805
6806In our example, we would want the following declarations:
6807
6808@example
6809%left '<'
6810%left '-'
6811%left '*'
6812@end example
6813
6814In a more complete example, which supports other operators as well, we
6815would declare them in groups of equal precedence. For example, @code{'+'} is
6816declared with @code{'-'}:
6817
6818@example
6819%left '<' '>' '=' NE LE GE
6820%left '+' '-'
6821%left '*' '/'
6822@end example
6823
6824@noindent
6825(Here @code{NE} and so on stand for the operators for ``not equal''
6826and so on. We assume that these tokens are more than one character long
6827and therefore are represented by names, not character literals.)
6828
6829@node How Precedence
6830@subsection How Precedence Works
6831
6832The first effect of the precedence declarations is to assign precedence
6833levels to the terminal symbols declared. The second effect is to assign
6834precedence levels to certain rules: each rule gets its precedence from
6835the last terminal symbol mentioned in the components. (You can also
6836specify explicitly the precedence of a rule. @xref{Contextual
6837Precedence, ,Context-Dependent Precedence}.)
6838
6839Finally, the resolution of conflicts works by comparing the precedence
6840of the rule being considered with that of the lookahead token. If the
6841token's precedence is higher, the choice is to shift. If the rule's
6842precedence is higher, the choice is to reduce. If they have equal
6843precedence, the choice is made based on the associativity of that
6844precedence level. The verbose output file made by @samp{-v}
6845(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
6846resolved.
6847
6848Not all rules and not all tokens have precedence. If either the rule or
6849the lookahead token has no precedence, then the default is to shift.
6850
6851@node Contextual Precedence
6852@section Context-Dependent Precedence
6853@cindex context-dependent precedence
6854@cindex unary operator precedence
6855@cindex precedence, context-dependent
6856@cindex precedence, unary operator
6857@findex %prec
6858
6859Often the precedence of an operator depends on the context. This sounds
6860outlandish at first, but it is really very common. For example, a minus
6861sign typically has a very high precedence as a unary operator, and a
6862somewhat lower precedence (lower than multiplication) as a binary operator.
6863
6864The Bison precedence declarations
6865can only be used once for a given token; so a token has
6866only one precedence declared in this way. For context-dependent
6867precedence, you need to use an additional mechanism: the @code{%prec}
6868modifier for rules.
6869
6870The @code{%prec} modifier declares the precedence of a particular rule by
6871specifying a terminal symbol whose precedence should be used for that rule.
6872It's not necessary for that symbol to appear otherwise in the rule. The
6873modifier's syntax is:
6874
6875@example
6876%prec @var{terminal-symbol}
6877@end example
6878
6879@noindent
6880and it is written after the components of the rule. Its effect is to
6881assign the rule the precedence of @var{terminal-symbol}, overriding
6882the precedence that would be deduced for it in the ordinary way. The
6883altered rule precedence then affects how conflicts involving that rule
6884are resolved (@pxref{Precedence, ,Operator Precedence}).
6885
6886Here is how @code{%prec} solves the problem of unary minus. First, declare
6887a precedence for a fictitious terminal symbol named @code{UMINUS}. There
6888are no tokens of this type, but the symbol serves to stand for its
6889precedence:
6890
6891@example
6892@dots{}
6893%left '+' '-'
6894%left '*'
6895%left UMINUS
6896@end example
6897
6898Now the precedence of @code{UMINUS} can be used in specific rules:
6899
6900@example
6901@group
6902exp: @dots{}
6903 | exp '-' exp
6904 @dots{}
6905 | '-' exp %prec UMINUS
6906@end group
6907@end example
6908
6909@ifset defaultprec
6910If you forget to append @code{%prec UMINUS} to the rule for unary
6911minus, Bison silently assumes that minus has its usual precedence.
6912This kind of problem can be tricky to debug, since one typically
6913discovers the mistake only by testing the code.
6914
6915The @code{%no-default-prec;} declaration makes it easier to discover
6916this kind of problem systematically. It causes rules that lack a
6917@code{%prec} modifier to have no precedence, even if the last terminal
6918symbol mentioned in their components has a declared precedence.
6919
6920If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
6921for all rules that participate in precedence conflict resolution.
6922Then you will see any shift/reduce conflict until you tell Bison how
6923to resolve it, either by changing your grammar or by adding an
6924explicit precedence. This will probably add declarations to the
6925grammar, but it helps to protect against incorrect rule precedences.
6926
6927The effect of @code{%no-default-prec;} can be reversed by giving
6928@code{%default-prec;}, which is the default.
6929@end ifset
6930
6931@node Parser States
6932@section Parser States
6933@cindex finite-state machine
6934@cindex parser state
6935@cindex state (of parser)
6936
6937The function @code{yyparse} is implemented using a finite-state machine.
6938The values pushed on the parser stack are not simply token type codes; they
6939represent the entire sequence of terminal and nonterminal symbols at or
6940near the top of the stack. The current state collects all the information
6941about previous input which is relevant to deciding what to do next.
6942
6943Each time a lookahead token is read, the current parser state together
6944with the type of lookahead token are looked up in a table. This table
6945entry can say, ``Shift the lookahead token.'' In this case, it also
6946specifies the new parser state, which is pushed onto the top of the
6947parser stack. Or it can say, ``Reduce using rule number @var{n}.''
6948This means that a certain number of tokens or groupings are taken off
6949the top of the stack, and replaced by one grouping. In other words,
6950that number of states are popped from the stack, and one new state is
6951pushed.
6952
6953There is one other alternative: the table can say that the lookahead token
6954is erroneous in the current state. This causes error processing to begin
6955(@pxref{Error Recovery}).
6956
6957@node Reduce/Reduce
6958@section Reduce/Reduce Conflicts
6959@cindex reduce/reduce conflict
6960@cindex conflicts, reduce/reduce
6961
6962A reduce/reduce conflict occurs if there are two or more rules that apply
6963to the same sequence of input. This usually indicates a serious error
6964in the grammar.
6965
6966For example, here is an erroneous attempt to define a sequence
6967of zero or more @code{word} groupings.
6968
6969@example
6970sequence: /* empty */
6971 @{ printf ("empty sequence\n"); @}
6972 | maybeword
6973 | sequence word
6974 @{ printf ("added word %s\n", $2); @}
6975 ;
6976
6977maybeword: /* empty */
6978 @{ printf ("empty maybeword\n"); @}
6979 | word
6980 @{ printf ("single word %s\n", $1); @}
6981 ;
6982@end example
6983
6984@noindent
6985The error is an ambiguity: there is more than one way to parse a single
6986@code{word} into a @code{sequence}. It could be reduced to a
6987@code{maybeword} and then into a @code{sequence} via the second rule.
6988Alternatively, nothing-at-all could be reduced into a @code{sequence}
6989via the first rule, and this could be combined with the @code{word}
6990using the third rule for @code{sequence}.
6991
6992There is also more than one way to reduce nothing-at-all into a
6993@code{sequence}. This can be done directly via the first rule,
6994or indirectly via @code{maybeword} and then the second rule.
6995
6996You might think that this is a distinction without a difference, because it
6997does not change whether any particular input is valid or not. But it does
6998affect which actions are run. One parsing order runs the second rule's
6999action; the other runs the first rule's action and the third rule's action.
7000In this example, the output of the program changes.
7001
7002Bison resolves a reduce/reduce conflict by choosing to use the rule that
7003appears first in the grammar, but it is very risky to rely on this. Every
7004reduce/reduce conflict must be studied and usually eliminated. Here is the
7005proper way to define @code{sequence}:
7006
7007@example
7008sequence: /* empty */
7009 @{ printf ("empty sequence\n"); @}
7010 | sequence word
7011 @{ printf ("added word %s\n", $2); @}
7012 ;
7013@end example
7014
7015Here is another common error that yields a reduce/reduce conflict:
7016
7017@example
7018sequence: /* empty */
7019 | sequence words
7020 | sequence redirects
7021 ;
7022
7023words: /* empty */
7024 | words word
7025 ;
7026
7027redirects:/* empty */
7028 | redirects redirect
7029 ;
7030@end example
7031
7032@noindent
7033The intention here is to define a sequence which can contain either
7034@code{word} or @code{redirect} groupings. The individual definitions of
7035@code{sequence}, @code{words} and @code{redirects} are error-free, but the
7036three together make a subtle ambiguity: even an empty input can be parsed
7037in infinitely many ways!
7038
7039Consider: nothing-at-all could be a @code{words}. Or it could be two
7040@code{words} in a row, or three, or any number. It could equally well be a
7041@code{redirects}, or two, or any number. Or it could be a @code{words}
7042followed by three @code{redirects} and another @code{words}. And so on.
7043
7044Here are two ways to correct these rules. First, to make it a single level
7045of sequence:
7046
7047@example
7048sequence: /* empty */
7049 | sequence word
7050 | sequence redirect
7051 ;
7052@end example
7053
7054Second, to prevent either a @code{words} or a @code{redirects}
7055from being empty:
7056
7057@example
7058sequence: /* empty */
7059 | sequence words
7060 | sequence redirects
7061 ;
7062
7063words: word
7064 | words word
7065 ;
7066
7067redirects:redirect
7068 | redirects redirect
7069 ;
7070@end example
7071
7072@node Mystery Conflicts
7073@section Mysterious Reduce/Reduce Conflicts
7074
7075Sometimes reduce/reduce conflicts can occur that don't look warranted.
7076Here is an example:
7077
7078@example
7079@group
7080%token ID
7081
7082%%
7083def: param_spec return_spec ','
7084 ;
7085param_spec:
7086 type
7087 | name_list ':' type
7088 ;
7089@end group
7090@group
7091return_spec:
7092 type
7093 | name ':' type
7094 ;
7095@end group
7096@group
7097type: ID
7098 ;
7099@end group
7100@group
7101name: ID
7102 ;
7103name_list:
7104 name
7105 | name ',' name_list
7106 ;
7107@end group
7108@end example
7109
7110It would seem that this grammar can be parsed with only a single token
7111of lookahead: when a @code{param_spec} is being read, an @code{ID} is
7112a @code{name} if a comma or colon follows, or a @code{type} if another
7113@code{ID} follows. In other words, this grammar is @acronym{LR}(1).
7114
7115@cindex @acronym{LR}(1)
7116@cindex @acronym{LALR}(1)
7117However, for historical reasons, Bison cannot by default handle all
7118@acronym{LR}(1) grammars.
7119In this grammar, two contexts, that after an @code{ID} at the beginning
7120of a @code{param_spec} and likewise at the beginning of a
7121@code{return_spec}, are similar enough that Bison assumes they are the
7122same.
7123They appear similar because the same set of rules would be
7124active---the rule for reducing to a @code{name} and that for reducing to
7125a @code{type}. Bison is unable to determine at that stage of processing
7126that the rules would require different lookahead tokens in the two
7127contexts, so it makes a single parser state for them both. Combining
7128the two contexts causes a conflict later. In parser terminology, this
7129occurrence means that the grammar is not @acronym{LALR}(1).
7130
7131For many practical grammars (specifically those that fall into the
7132non-@acronym{LR}(1) class), the limitations of @acronym{LALR}(1) result in
7133difficulties beyond just mysterious reduce/reduce conflicts.
7134The best way to fix all these problems is to select a different parser
7135table generation algorithm.
7136Either @acronym{IELR}(1) or canonical @acronym{LR}(1) would suffice, but
7137the former is more efficient and easier to debug during development.
7138@xref{Decl Summary,,lr.type}, for details.
7139(Bison's @acronym{IELR}(1) and canonical @acronym{LR}(1) implementations
7140are experimental.
7141More user feedback will help to stabilize them.)
7142
7143If you instead wish to work around @acronym{LALR}(1)'s limitations, you
7144can often fix a mysterious conflict by identifying the two parser states
7145that are being confused, and adding something to make them look
7146distinct. In the above example, adding one rule to
7147@code{return_spec} as follows makes the problem go away:
7148
7149@example
7150@group
7151%token BOGUS
7152@dots{}
7153%%
7154@dots{}
7155return_spec:
7156 type
7157 | name ':' type
7158 /* This rule is never used. */
7159 | ID BOGUS
7160 ;
7161@end group
7162@end example
7163
7164This corrects the problem because it introduces the possibility of an
7165additional active rule in the context after the @code{ID} at the beginning of
7166@code{return_spec}. This rule is not active in the corresponding context
7167in a @code{param_spec}, so the two contexts receive distinct parser states.
7168As long as the token @code{BOGUS} is never generated by @code{yylex},
7169the added rule cannot alter the way actual input is parsed.
7170
7171In this particular example, there is another way to solve the problem:
7172rewrite the rule for @code{return_spec} to use @code{ID} directly
7173instead of via @code{name}. This also causes the two confusing
7174contexts to have different sets of active rules, because the one for
7175@code{return_spec} activates the altered rule for @code{return_spec}
7176rather than the one for @code{name}.
7177
7178@example
7179param_spec:
7180 type
7181 | name_list ':' type
7182 ;
7183return_spec:
7184 type
7185 | ID ':' type
7186 ;
7187@end example
7188
7189For a more detailed exposition of @acronym{LALR}(1) parsers and parser
7190generators, please see:
7191Frank DeRemer and Thomas Pennello, Efficient Computation of
7192@acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
7193Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
7194pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
7195
7196@node Generalized LR Parsing
7197@section Generalized @acronym{LR} (@acronym{GLR}) Parsing
7198@cindex @acronym{GLR} parsing
7199@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
7200@cindex ambiguous grammars
7201@cindex nondeterministic parsing
7202
7203Bison produces @emph{deterministic} parsers that choose uniquely
7204when to reduce and which reduction to apply
7205based on a summary of the preceding input and on one extra token of lookahead.
7206As a result, normal Bison handles a proper subset of the family of
7207context-free languages.
7208Ambiguous grammars, since they have strings with more than one possible
7209sequence of reductions cannot have deterministic parsers in this sense.
7210The same is true of languages that require more than one symbol of
7211lookahead, since the parser lacks the information necessary to make a
7212decision at the point it must be made in a shift-reduce parser.
7213Finally, as previously mentioned (@pxref{Mystery Conflicts}),
7214there are languages where Bison's default choice of how to
7215summarize the input seen so far loses necessary information.
7216
7217When you use the @samp{%glr-parser} declaration in your grammar file,
7218Bison generates a parser that uses a different algorithm, called
7219Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
7220parser uses the same basic
7221algorithm for parsing as an ordinary Bison parser, but behaves
7222differently in cases where there is a shift-reduce conflict that has not
7223been resolved by precedence rules (@pxref{Precedence}) or a
7224reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
7225situation, it
7226effectively @emph{splits} into a several parsers, one for each possible
7227shift or reduction. These parsers then proceed as usual, consuming
7228tokens in lock-step. Some of the stacks may encounter other conflicts
7229and split further, with the result that instead of a sequence of states,
7230a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
7231
7232In effect, each stack represents a guess as to what the proper parse
7233is. Additional input may indicate that a guess was wrong, in which case
7234the appropriate stack silently disappears. Otherwise, the semantics
7235actions generated in each stack are saved, rather than being executed
7236immediately. When a stack disappears, its saved semantic actions never
7237get executed. When a reduction causes two stacks to become equivalent,
7238their sets of semantic actions are both saved with the state that
7239results from the reduction. We say that two stacks are equivalent
7240when they both represent the same sequence of states,
7241and each pair of corresponding states represents a
7242grammar symbol that produces the same segment of the input token
7243stream.
7244
7245Whenever the parser makes a transition from having multiple
7246states to having one, it reverts to the normal deterministic parsing
7247algorithm, after resolving and executing the saved-up actions.
7248At this transition, some of the states on the stack will have semantic
7249values that are sets (actually multisets) of possible actions. The
7250parser tries to pick one of the actions by first finding one whose rule
7251has the highest dynamic precedence, as set by the @samp{%dprec}
7252declaration. Otherwise, if the alternative actions are not ordered by
7253precedence, but there the same merging function is declared for both
7254rules by the @samp{%merge} declaration,
7255Bison resolves and evaluates both and then calls the merge function on
7256the result. Otherwise, it reports an ambiguity.
7257
7258It is possible to use a data structure for the @acronym{GLR} parsing tree that
7259permits the processing of any @acronym{LR}(1) grammar in linear time (in the
7260size of the input), any unambiguous (not necessarily
7261@acronym{LR}(1)) grammar in
7262quadratic worst-case time, and any general (possibly ambiguous)
7263context-free grammar in cubic worst-case time. However, Bison currently
7264uses a simpler data structure that requires time proportional to the
7265length of the input times the maximum number of stacks required for any
7266prefix of the input. Thus, really ambiguous or nondeterministic
7267grammars can require exponential time and space to process. Such badly
7268behaving examples, however, are not generally of practical interest.
7269Usually, nondeterminism in a grammar is local---the parser is ``in
7270doubt'' only for a few tokens at a time. Therefore, the current data
7271structure should generally be adequate. On @acronym{LR}(1) portions of a
7272grammar, in particular, it is only slightly slower than with the
7273deterministic @acronym{LR}(1) Bison parser.
7274
7275For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
7276Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
7277Generalised @acronym{LR} Parsers, Royal Holloway, University of
7278London, Department of Computer Science, TR-00-12,
7279@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
7280(2000-12-24).
7281
7282@node Memory Management
7283@section Memory Management, and How to Avoid Memory Exhaustion
7284@cindex memory exhaustion
7285@cindex memory management
7286@cindex stack overflow
7287@cindex parser stack overflow
7288@cindex overflow of parser stack
7289
7290The Bison parser stack can run out of memory if too many tokens are shifted and
7291not reduced. When this happens, the parser function @code{yyparse}
7292calls @code{yyerror} and then returns 2.
7293
7294Because Bison parsers have growing stacks, hitting the upper limit
7295usually results from using a right recursion instead of a left
7296recursion, @xref{Recursion, ,Recursive Rules}.
7297
7298@vindex YYMAXDEPTH
7299By defining the macro @code{YYMAXDEPTH}, you can control how deep the
7300parser stack can become before memory is exhausted. Define the
7301macro with a value that is an integer. This value is the maximum number
7302of tokens that can be shifted (and not reduced) before overflow.
7303
7304The stack space allowed is not necessarily allocated. If you specify a
7305large value for @code{YYMAXDEPTH}, the parser normally allocates a small
7306stack at first, and then makes it bigger by stages as needed. This
7307increasing allocation happens automatically and silently. Therefore,
7308you do not need to make @code{YYMAXDEPTH} painfully small merely to save
7309space for ordinary inputs that do not need much stack.
7310
7311However, do not allow @code{YYMAXDEPTH} to be a value so large that
7312arithmetic overflow could occur when calculating the size of the stack
7313space. Also, do not allow @code{YYMAXDEPTH} to be less than
7314@code{YYINITDEPTH}.
7315
7316@cindex default stack limit
7317The default value of @code{YYMAXDEPTH}, if you do not define it, is
731810000.
7319
7320@vindex YYINITDEPTH
7321You can control how much stack is allocated initially by defining the
7322macro @code{YYINITDEPTH} to a positive integer. For the deterministic
7323parser in C, this value must be a compile-time constant
7324unless you are assuming C99 or some other target language or compiler
7325that allows variable-length arrays. The default is 200.
7326
7327Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
7328
7329@c FIXME: C++ output.
7330Because of semantic differences between C and C++, the deterministic
7331parsers in C produced by Bison cannot grow when compiled
7332by C++ compilers. In this precise case (compiling a C parser as C++) you are
7333suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
7334this deficiency in a future release.
7335
7336@node Error Recovery
7337@chapter Error Recovery
7338@cindex error recovery
7339@cindex recovery from errors
7340
7341It is not usually acceptable to have a program terminate on a syntax
7342error. For example, a compiler should recover sufficiently to parse the
7343rest of the input file and check it for errors; a calculator should accept
7344another expression.
7345
7346In a simple interactive command parser where each input is one line, it may
7347be sufficient to allow @code{yyparse} to return 1 on error and have the
7348caller ignore the rest of the input line when that happens (and then call
7349@code{yyparse} again). But this is inadequate for a compiler, because it
7350forgets all the syntactic context leading up to the error. A syntax error
7351deep within a function in the compiler input should not cause the compiler
7352to treat the following line like the beginning of a source file.
7353
7354@findex error
7355You can define how to recover from a syntax error by writing rules to
7356recognize the special token @code{error}. This is a terminal symbol that
7357is always defined (you need not declare it) and reserved for error
7358handling. The Bison parser generates an @code{error} token whenever a
7359syntax error happens; if you have provided a rule to recognize this token
7360in the current context, the parse can continue.
7361
7362For example:
7363
7364@example
7365stmnts: /* empty string */
7366 | stmnts '\n'
7367 | stmnts exp '\n'
7368 | stmnts error '\n'
7369@end example
7370
7371The fourth rule in this example says that an error followed by a newline
7372makes a valid addition to any @code{stmnts}.
7373
7374What happens if a syntax error occurs in the middle of an @code{exp}? The
7375error recovery rule, interpreted strictly, applies to the precise sequence
7376of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
7377the middle of an @code{exp}, there will probably be some additional tokens
7378and subexpressions on the stack after the last @code{stmnts}, and there
7379will be tokens to read before the next newline. So the rule is not
7380applicable in the ordinary way.
7381
7382But Bison can force the situation to fit the rule, by discarding part of
7383the semantic context and part of the input. First it discards states
7384and objects from the stack until it gets back to a state in which the
7385@code{error} token is acceptable. (This means that the subexpressions
7386already parsed are discarded, back to the last complete @code{stmnts}.)
7387At this point the @code{error} token can be shifted. Then, if the old
7388lookahead token is not acceptable to be shifted next, the parser reads
7389tokens and discards them until it finds a token which is acceptable. In
7390this example, Bison reads and discards input until the next newline so
7391that the fourth rule can apply. Note that discarded symbols are
7392possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
7393Discarded Symbols}, for a means to reclaim this memory.
7394
7395The choice of error rules in the grammar is a choice of strategies for
7396error recovery. A simple and useful strategy is simply to skip the rest of
7397the current input line or current statement if an error is detected:
7398
7399@example
7400stmnt: error ';' /* On error, skip until ';' is read. */
7401@end example
7402
7403It is also useful to recover to the matching close-delimiter of an
7404opening-delimiter that has already been parsed. Otherwise the
7405close-delimiter will probably appear to be unmatched, and generate another,
7406spurious error message:
7407
7408@example
7409primary: '(' expr ')'
7410 | '(' error ')'
7411 @dots{}
7412 ;
7413@end example
7414
7415Error recovery strategies are necessarily guesses. When they guess wrong,
7416one syntax error often leads to another. In the above example, the error
7417recovery rule guesses that an error is due to bad input within one
7418@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
7419middle of a valid @code{stmnt}. After the error recovery rule recovers
7420from the first error, another syntax error will be found straightaway,
7421since the text following the spurious semicolon is also an invalid
7422@code{stmnt}.
7423
7424To prevent an outpouring of error messages, the parser will output no error
7425message for another syntax error that happens shortly after the first; only
7426after three consecutive input tokens have been successfully shifted will
7427error messages resume.
7428
7429Note that rules which accept the @code{error} token may have actions, just
7430as any other rules can.
7431
7432@findex yyerrok
7433You can make error messages resume immediately by using the macro
7434@code{yyerrok} in an action. If you do this in the error rule's action, no
7435error messages will be suppressed. This macro requires no arguments;
7436@samp{yyerrok;} is a valid C statement.
7437
7438@findex yyclearin
7439The previous lookahead token is reanalyzed immediately after an error. If
7440this is unacceptable, then the macro @code{yyclearin} may be used to clear
7441this token. Write the statement @samp{yyclearin;} in the error rule's
7442action.
7443@xref{Action Features, ,Special Features for Use in Actions}.
7444
7445For example, suppose that on a syntax error, an error handling routine is
7446called that advances the input stream to some point where parsing should
7447once again commence. The next symbol returned by the lexical scanner is
7448probably correct. The previous lookahead token ought to be discarded
7449with @samp{yyclearin;}.
7450
7451@vindex YYRECOVERING
7452The expression @code{YYRECOVERING ()} yields 1 when the parser
7453is recovering from a syntax error, and 0 otherwise.
7454Syntax error diagnostics are suppressed while recovering from a syntax
7455error.
7456
7457@node Context Dependency
7458@chapter Handling Context Dependencies
7459
7460The Bison paradigm is to parse tokens first, then group them into larger
7461syntactic units. In many languages, the meaning of a token is affected by
7462its context. Although this violates the Bison paradigm, certain techniques
7463(known as @dfn{kludges}) may enable you to write Bison parsers for such
7464languages.
7465
7466@menu
7467* Semantic Tokens:: Token parsing can depend on the semantic context.
7468* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
7469* Tie-in Recovery:: Lexical tie-ins have implications for how
7470 error recovery rules must be written.
7471@end menu
7472
7473(Actually, ``kludge'' means any technique that gets its job done but is
7474neither clean nor robust.)
7475
7476@node Semantic Tokens
7477@section Semantic Info in Token Types
7478
7479The C language has a context dependency: the way an identifier is used
7480depends on what its current meaning is. For example, consider this:
7481
7482@example
7483foo (x);
7484@end example
7485
7486This looks like a function call statement, but if @code{foo} is a typedef
7487name, then this is actually a declaration of @code{x}. How can a Bison
7488parser for C decide how to parse this input?
7489
7490The method used in @acronym{GNU} C is to have two different token types,
7491@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
7492identifier, it looks up the current declaration of the identifier in order
7493to decide which token type to return: @code{TYPENAME} if the identifier is
7494declared as a typedef, @code{IDENTIFIER} otherwise.
7495
7496The grammar rules can then express the context dependency by the choice of
7497token type to recognize. @code{IDENTIFIER} is accepted as an expression,
7498but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
7499@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
7500is @emph{not} significant, such as in declarations that can shadow a
7501typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
7502accepted---there is one rule for each of the two token types.
7503
7504This technique is simple to use if the decision of which kinds of
7505identifiers to allow is made at a place close to where the identifier is
7506parsed. But in C this is not always so: C allows a declaration to
7507redeclare a typedef name provided an explicit type has been specified
7508earlier:
7509
7510@example
7511typedef int foo, bar;
7512int baz (void)
7513@{
7514 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
7515 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
7516 return foo (bar);
7517@}
7518@end example
7519
7520Unfortunately, the name being declared is separated from the declaration
7521construct itself by a complicated syntactic structure---the ``declarator''.
7522
7523As a result, part of the Bison parser for C needs to be duplicated, with
7524all the nonterminal names changed: once for parsing a declaration in
7525which a typedef name can be redefined, and once for parsing a
7526declaration in which that can't be done. Here is a part of the
7527duplication, with actions omitted for brevity:
7528
7529@example
7530initdcl:
7531 declarator maybeasm '='
7532 init
7533 | declarator maybeasm
7534 ;
7535
7536notype_initdcl:
7537 notype_declarator maybeasm '='
7538 init
7539 | notype_declarator maybeasm
7540 ;
7541@end example
7542
7543@noindent
7544Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
7545cannot. The distinction between @code{declarator} and
7546@code{notype_declarator} is the same sort of thing.
7547
7548There is some similarity between this technique and a lexical tie-in
7549(described next), in that information which alters the lexical analysis is
7550changed during parsing by other parts of the program. The difference is
7551here the information is global, and is used for other purposes in the
7552program. A true lexical tie-in has a special-purpose flag controlled by
7553the syntactic context.
7554
7555@node Lexical Tie-ins
7556@section Lexical Tie-ins
7557@cindex lexical tie-in
7558
7559One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7560which is set by Bison actions, whose purpose is to alter the way tokens are
7561parsed.
7562
7563For example, suppose we have a language vaguely like C, but with a special
7564construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7565an expression in parentheses in which all integers are hexadecimal. In
7566particular, the token @samp{a1b} must be treated as an integer rather than
7567as an identifier if it appears in that context. Here is how you can do it:
7568
7569@example
7570@group
7571%@{
7572 int hexflag;
7573 int yylex (void);
7574 void yyerror (char const *);
7575%@}
7576%%
7577@dots{}
7578@end group
7579@group
7580expr: IDENTIFIER
7581 | constant
7582 | HEX '('
7583 @{ hexflag = 1; @}
7584 expr ')'
7585 @{ hexflag = 0;
7586 $$ = $4; @}
7587 | expr '+' expr
7588 @{ $$ = make_sum ($1, $3); @}
7589 @dots{}
7590 ;
7591@end group
7592
7593@group
7594constant:
7595 INTEGER
7596 | STRING
7597 ;
7598@end group
7599@end example
7600
7601@noindent
7602Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
7603it is nonzero, all integers are parsed in hexadecimal, and tokens starting
7604with letters are parsed as integers if possible.
7605
7606The declaration of @code{hexflag} shown in the prologue of the parser file
7607is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
7608You must also write the code in @code{yylex} to obey the flag.
7609
7610@node Tie-in Recovery
7611@section Lexical Tie-ins and Error Recovery
7612
7613Lexical tie-ins make strict demands on any error recovery rules you have.
7614@xref{Error Recovery}.
7615
7616The reason for this is that the purpose of an error recovery rule is to
7617abort the parsing of one construct and resume in some larger construct.
7618For example, in C-like languages, a typical error recovery rule is to skip
7619tokens until the next semicolon, and then start a new statement, like this:
7620
7621@example
7622stmt: expr ';'
7623 | IF '(' expr ')' stmt @{ @dots{} @}
7624 @dots{}
7625 error ';'
7626 @{ hexflag = 0; @}
7627 ;
7628@end example
7629
7630If there is a syntax error in the middle of a @samp{hex (@var{expr})}
7631construct, this error rule will apply, and then the action for the
7632completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
7633remain set for the entire rest of the input, or until the next @code{hex}
7634keyword, causing identifiers to be misinterpreted as integers.
7635
7636To avoid this problem the error recovery rule itself clears @code{hexflag}.
7637
7638There may also be an error recovery rule that works within expressions.
7639For example, there could be a rule which applies within parentheses
7640and skips to the close-parenthesis:
7641
7642@example
7643@group
7644expr: @dots{}
7645 | '(' expr ')'
7646 @{ $$ = $2; @}
7647 | '(' error ')'
7648 @dots{}
7649@end group
7650@end example
7651
7652If this rule acts within the @code{hex} construct, it is not going to abort
7653that construct (since it applies to an inner level of parentheses within
7654the construct). Therefore, it should not clear the flag: the rest of
7655the @code{hex} construct should be parsed with the flag still in effect.
7656
7657What if there is an error recovery rule which might abort out of the
7658@code{hex} construct or might not, depending on circumstances? There is no
7659way you can write the action to determine whether a @code{hex} construct is
7660being aborted or not. So if you are using a lexical tie-in, you had better
7661make sure your error recovery rules are not of this kind. Each rule must
7662be such that you can be sure that it always will, or always won't, have to
7663clear the flag.
7664
7665@c ================================================== Debugging Your Parser
7666
7667@node Debugging
7668@chapter Debugging Your Parser
7669
7670Developing a parser can be a challenge, especially if you don't
7671understand the algorithm (@pxref{Algorithm, ,The Bison Parser
7672Algorithm}). Even so, sometimes a detailed description of the automaton
7673can help (@pxref{Understanding, , Understanding Your Parser}), or
7674tracing the execution of the parser can give some insight on why it
7675behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
7676
7677@menu
7678* Understanding:: Understanding the structure of your parser.
7679* Tracing:: Tracing the execution of your parser.
7680@end menu
7681
7682@node Understanding
7683@section Understanding Your Parser
7684
7685As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
7686Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
7687frequent than one would hope), looking at this automaton is required to
7688tune or simply fix a parser. Bison provides two different
7689representation of it, either textually or graphically (as a DOT file).
7690
7691The textual file is generated when the options @option{--report} or
7692@option{--verbose} are specified, see @xref{Invocation, , Invoking
7693Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
7694the parser output file name, and adding @samp{.output} instead.
7695Therefore, if the input file is @file{foo.y}, then the parser file is
7696called @file{foo.tab.c} by default. As a consequence, the verbose
7697output file is called @file{foo.output}.
7698
7699The following grammar file, @file{calc.y}, will be used in the sequel:
7700
7701@example
7702%token NUM STR
7703%left '+' '-'
7704%left '*'
7705%%
7706exp: exp '+' exp
7707 | exp '-' exp
7708 | exp '*' exp
7709 | exp '/' exp
7710 | NUM
7711 ;
7712useless: STR;
7713%%
7714@end example
7715
7716@command{bison} reports:
7717
7718@example
7719calc.y: warning: 1 nonterminal useless in grammar
7720calc.y: warning: 1 rule useless in grammar
7721calc.y:11.1-7: warning: nonterminal useless in grammar: useless
7722calc.y:11.10-12: warning: rule useless in grammar: useless: STR
7723calc.y: conflicts: 7 shift/reduce
7724@end example
7725
7726When given @option{--report=state}, in addition to @file{calc.tab.c}, it
7727creates a file @file{calc.output} with contents detailed below. The
7728order of the output and the exact presentation might vary, but the
7729interpretation is the same.
7730
7731The first section includes details on conflicts that were solved thanks
7732to precedence and/or associativity:
7733
7734@example
7735Conflict in state 8 between rule 2 and token '+' resolved as reduce.
7736Conflict in state 8 between rule 2 and token '-' resolved as reduce.
7737Conflict in state 8 between rule 2 and token '*' resolved as shift.
7738@exdent @dots{}
7739@end example
7740
7741@noindent
7742The next section lists states that still have conflicts.
7743
7744@example
7745State 8 conflicts: 1 shift/reduce
7746State 9 conflicts: 1 shift/reduce
7747State 10 conflicts: 1 shift/reduce
7748State 11 conflicts: 4 shift/reduce
7749@end example
7750
7751@noindent
7752@cindex token, useless
7753@cindex useless token
7754@cindex nonterminal, useless
7755@cindex useless nonterminal
7756@cindex rule, useless
7757@cindex useless rule
7758The next section reports useless tokens, nonterminal and rules. Useless
7759nonterminals and rules are removed in order to produce a smaller parser,
7760but useless tokens are preserved, since they might be used by the
7761scanner (note the difference between ``useless'' and ``unused''
7762below):
7763
7764@example
7765Nonterminals useless in grammar:
7766 useless
7767
7768Terminals unused in grammar:
7769 STR
7770
7771Rules useless in grammar:
7772#6 useless: STR;
7773@end example
7774
7775@noindent
7776The next section reproduces the exact grammar that Bison used:
7777
7778@example
7779Grammar
7780
7781 Number, Line, Rule
7782 0 5 $accept -> exp $end
7783 1 5 exp -> exp '+' exp
7784 2 6 exp -> exp '-' exp
7785 3 7 exp -> exp '*' exp
7786 4 8 exp -> exp '/' exp
7787 5 9 exp -> NUM
7788@end example
7789
7790@noindent
7791and reports the uses of the symbols:
7792
7793@example
7794Terminals, with rules where they appear
7795
7796$end (0) 0
7797'*' (42) 3
7798'+' (43) 1
7799'-' (45) 2
7800'/' (47) 4
7801error (256)
7802NUM (258) 5
7803
7804Nonterminals, with rules where they appear
7805
7806$accept (8)
7807 on left: 0
7808exp (9)
7809 on left: 1 2 3 4 5, on right: 0 1 2 3 4
7810@end example
7811
7812@noindent
7813@cindex item
7814@cindex pointed rule
7815@cindex rule, pointed
7816Bison then proceeds onto the automaton itself, describing each state
7817with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
7818item is a production rule together with a point (marked by @samp{.})
7819that the input cursor.
7820
7821@example
7822state 0
7823
7824 $accept -> . exp $ (rule 0)
7825
7826 NUM shift, and go to state 1
7827
7828 exp go to state 2
7829@end example
7830
7831This reads as follows: ``state 0 corresponds to being at the very
7832beginning of the parsing, in the initial rule, right before the start
7833symbol (here, @code{exp}). When the parser returns to this state right
7834after having reduced a rule that produced an @code{exp}, the control
7835flow jumps to state 2. If there is no such transition on a nonterminal
7836symbol, and the lookahead is a @code{NUM}, then this token is shifted on
7837the parse stack, and the control flow jumps to state 1. Any other
7838lookahead triggers a syntax error.''
7839
7840@cindex core, item set
7841@cindex item set core
7842@cindex kernel, item set
7843@cindex item set core
7844Even though the only active rule in state 0 seems to be rule 0, the
7845report lists @code{NUM} as a lookahead token because @code{NUM} can be
7846at the beginning of any rule deriving an @code{exp}. By default Bison
7847reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
7848you want to see more detail you can invoke @command{bison} with
7849@option{--report=itemset} to list all the items, include those that can
7850be derived:
7851
7852@example
7853state 0
7854
7855 $accept -> . exp $ (rule 0)
7856 exp -> . exp '+' exp (rule 1)
7857 exp -> . exp '-' exp (rule 2)
7858 exp -> . exp '*' exp (rule 3)
7859 exp -> . exp '/' exp (rule 4)
7860 exp -> . NUM (rule 5)
7861
7862 NUM shift, and go to state 1
7863
7864 exp go to state 2
7865@end example
7866
7867@noindent
7868In the state 1...
7869
7870@example
7871state 1
7872
7873 exp -> NUM . (rule 5)
7874
7875 $default reduce using rule 5 (exp)
7876@end example
7877
7878@noindent
7879the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
7880(@samp{$default}), the parser will reduce it. If it was coming from
7881state 0, then, after this reduction it will return to state 0, and will
7882jump to state 2 (@samp{exp: go to state 2}).
7883
7884@example
7885state 2
7886
7887 $accept -> exp . $ (rule 0)
7888 exp -> exp . '+' exp (rule 1)
7889 exp -> exp . '-' exp (rule 2)
7890 exp -> exp . '*' exp (rule 3)
7891 exp -> exp . '/' exp (rule 4)
7892
7893 $ shift, and go to state 3
7894 '+' shift, and go to state 4
7895 '-' shift, and go to state 5
7896 '*' shift, and go to state 6
7897 '/' shift, and go to state 7
7898@end example
7899
7900@noindent
7901In state 2, the automaton can only shift a symbol. For instance,
7902because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
7903@samp{+}, it will be shifted on the parse stack, and the automaton
7904control will jump to state 4, corresponding to the item @samp{exp -> exp
7905'+' . exp}. Since there is no default action, any other token than
7906those listed above will trigger a syntax error.
7907
7908@cindex accepting state
7909The state 3 is named the @dfn{final state}, or the @dfn{accepting
7910state}:
7911
7912@example
7913state 3
7914
7915 $accept -> exp $ . (rule 0)
7916
7917 $default accept
7918@end example
7919
7920@noindent
7921the initial rule is completed (the start symbol and the end
7922of input were read), the parsing exits successfully.
7923
7924The interpretation of states 4 to 7 is straightforward, and is left to
7925the reader.
7926
7927@example
7928state 4
7929
7930 exp -> exp '+' . exp (rule 1)
7931
7932 NUM shift, and go to state 1
7933
7934 exp go to state 8
7935
7936state 5
7937
7938 exp -> exp '-' . exp (rule 2)
7939
7940 NUM shift, and go to state 1
7941
7942 exp go to state 9
7943
7944state 6
7945
7946 exp -> exp '*' . exp (rule 3)
7947
7948 NUM shift, and go to state 1
7949
7950 exp go to state 10
7951
7952state 7
7953
7954 exp -> exp '/' . exp (rule 4)
7955
7956 NUM shift, and go to state 1
7957
7958 exp go to state 11
7959@end example
7960
7961As was announced in beginning of the report, @samp{State 8 conflicts:
79621 shift/reduce}:
7963
7964@example
7965state 8
7966
7967 exp -> exp . '+' exp (rule 1)
7968 exp -> exp '+' exp . (rule 1)
7969 exp -> exp . '-' exp (rule 2)
7970 exp -> exp . '*' exp (rule 3)
7971 exp -> exp . '/' exp (rule 4)
7972
7973 '*' shift, and go to state 6
7974 '/' shift, and go to state 7
7975
7976 '/' [reduce using rule 1 (exp)]
7977 $default reduce using rule 1 (exp)
7978@end example
7979
7980Indeed, there are two actions associated to the lookahead @samp{/}:
7981either shifting (and going to state 7), or reducing rule 1. The
7982conflict means that either the grammar is ambiguous, or the parser lacks
7983information to make the right decision. Indeed the grammar is
7984ambiguous, as, since we did not specify the precedence of @samp{/}, the
7985sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
7986NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
7987NUM}, which corresponds to reducing rule 1.
7988
7989Because in deterministic parsing a single decision can be made, Bison
7990arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
7991Shift/Reduce Conflicts}. Discarded actions are reported in between
7992square brackets.
7993
7994Note that all the previous states had a single possible action: either
7995shifting the next token and going to the corresponding state, or
7996reducing a single rule. In the other cases, i.e., when shifting
7997@emph{and} reducing is possible or when @emph{several} reductions are
7998possible, the lookahead is required to select the action. State 8 is
7999one such state: if the lookahead is @samp{*} or @samp{/} then the action
8000is shifting, otherwise the action is reducing rule 1. In other words,
8001the first two items, corresponding to rule 1, are not eligible when the
8002lookahead token is @samp{*}, since we specified that @samp{*} has higher
8003precedence than @samp{+}. More generally, some items are eligible only
8004with some set of possible lookahead tokens. When run with
8005@option{--report=lookahead}, Bison specifies these lookahead tokens:
8006
8007@example
8008state 8
8009
8010 exp -> exp . '+' exp (rule 1)
8011 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
8012 exp -> exp . '-' exp (rule 2)
8013 exp -> exp . '*' exp (rule 3)
8014 exp -> exp . '/' exp (rule 4)
8015
8016 '*' shift, and go to state 6
8017 '/' shift, and go to state 7
8018
8019 '/' [reduce using rule 1 (exp)]
8020 $default reduce using rule 1 (exp)
8021@end example
8022
8023The remaining states are similar:
8024
8025@example
8026state 9
8027
8028 exp -> exp . '+' exp (rule 1)
8029 exp -> exp . '-' exp (rule 2)
8030 exp -> exp '-' exp . (rule 2)
8031 exp -> exp . '*' exp (rule 3)
8032 exp -> exp . '/' exp (rule 4)
8033
8034 '*' shift, and go to state 6
8035 '/' shift, and go to state 7
8036
8037 '/' [reduce using rule 2 (exp)]
8038 $default reduce using rule 2 (exp)
8039
8040state 10
8041
8042 exp -> exp . '+' exp (rule 1)
8043 exp -> exp . '-' exp (rule 2)
8044 exp -> exp . '*' exp (rule 3)
8045 exp -> exp '*' exp . (rule 3)
8046 exp -> exp . '/' exp (rule 4)
8047
8048 '/' shift, and go to state 7
8049
8050 '/' [reduce using rule 3 (exp)]
8051 $default reduce using rule 3 (exp)
8052
8053state 11
8054
8055 exp -> exp . '+' exp (rule 1)
8056 exp -> exp . '-' exp (rule 2)
8057 exp -> exp . '*' exp (rule 3)
8058 exp -> exp . '/' exp (rule 4)
8059 exp -> exp '/' exp . (rule 4)
8060
8061 '+' shift, and go to state 4
8062 '-' shift, and go to state 5
8063 '*' shift, and go to state 6
8064 '/' shift, and go to state 7
8065
8066 '+' [reduce using rule 4 (exp)]
8067 '-' [reduce using rule 4 (exp)]
8068 '*' [reduce using rule 4 (exp)]
8069 '/' [reduce using rule 4 (exp)]
8070 $default reduce using rule 4 (exp)
8071@end example
8072
8073@noindent
8074Observe that state 11 contains conflicts not only due to the lack of
8075precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
8076@samp{*}, but also because the
8077associativity of @samp{/} is not specified.
8078
8079
8080@node Tracing
8081@section Tracing Your Parser
8082@findex yydebug
8083@cindex debugging
8084@cindex tracing the parser
8085
8086If a Bison grammar compiles properly but doesn't do what you want when it
8087runs, the @code{yydebug} parser-trace feature can help you figure out why.
8088
8089There are several means to enable compilation of trace facilities:
8090
8091@table @asis
8092@item the macro @code{YYDEBUG}
8093@findex YYDEBUG
8094Define the macro @code{YYDEBUG} to a nonzero value when you compile the
8095parser. This is compliant with @acronym{POSIX} Yacc. You could use
8096@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
8097YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
8098Prologue}).
8099
8100@item the option @option{-t}, @option{--debug}
8101Use the @samp{-t} option when you run Bison (@pxref{Invocation,
8102,Invoking Bison}). This is @acronym{POSIX} compliant too.
8103
8104@item the directive @samp{%debug}
8105@findex %debug
8106Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison Declaration
8107Summary}). This Bison extension is maintained for backward
8108compatibility with previous versions of Bison.
8109
8110@item the variable @samp{parse.trace}
8111@findex %define parse.trace
8112Add the @samp{%define parse.trace} directive (@pxref{Decl Summary,
8113,Bison Declaration Summary}), or pass the @option{-Dparse.trace} option
8114(@pxref{Bison Options}). This is a Bison extension, which is especially
8115useful for languages that don't use a preprocessor. Unless
8116@acronym{POSIX} and Yacc portability matter to you, this is the
8117preferred solution.
8118@end table
8119
8120We suggest that you always enable the trace option so that debugging is
8121always possible.
8122
8123The trace facility outputs messages with macro calls of the form
8124@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
8125@var{format} and @var{args} are the usual @code{printf} format and variadic
8126arguments. If you define @code{YYDEBUG} to a nonzero value but do not
8127define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
8128and @code{YYFPRINTF} is defined to @code{fprintf}.
8129
8130Once you have compiled the program with trace facilities, the way to
8131request a trace is to store a nonzero value in the variable @code{yydebug}.
8132You can do this by making the C code do it (in @code{main}, perhaps), or
8133you can alter the value with a C debugger.
8134
8135Each step taken by the parser when @code{yydebug} is nonzero produces a
8136line or two of trace information, written on @code{stderr}. The trace
8137messages tell you these things:
8138
8139@itemize @bullet
8140@item
8141Each time the parser calls @code{yylex}, what kind of token was read.
8142
8143@item
8144Each time a token is shifted, the depth and complete contents of the
8145state stack (@pxref{Parser States}).
8146
8147@item
8148Each time a rule is reduced, which rule it is, and the complete contents
8149of the state stack afterward.
8150@end itemize
8151
8152To make sense of this information, it helps to refer to the listing file
8153produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
8154Bison}). This file shows the meaning of each state in terms of
8155positions in various rules, and also what each state will do with each
8156possible input token. As you read the successive trace messages, you
8157can see that the parser is functioning according to its specification in
8158the listing file. Eventually you will arrive at the place where
8159something undesirable happens, and you will see which parts of the
8160grammar are to blame.
8161
8162The parser file is a C program and you can use C debuggers on it, but it's
8163not easy to interpret what it is doing. The parser function is a
8164finite-state machine interpreter, and aside from the actions it executes
8165the same code over and over. Only the values of variables show where in
8166the grammar it is working.
8167
8168@findex YYPRINT
8169The debugging information normally gives the token type of each token
8170read, but not its semantic value. You can optionally define a macro
8171named @code{YYPRINT} to provide a way to print the value. If you define
8172@code{YYPRINT}, it should take three arguments. The parser will pass a
8173standard I/O stream, the numeric code for the token type, and the token
8174value (from @code{yylval}).
8175
8176Here is an example of @code{YYPRINT} suitable for the multi-function
8177calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}):
8178
8179@smallexample
8180%@{
8181 static void print_token_value (FILE *, int, YYSTYPE);
8182 #define YYPRINT(file, type, value) print_token_value (file, type, value)
8183%@}
8184
8185@dots{} %% @dots{} %% @dots{}
8186
8187static void
8188print_token_value (FILE *file, int type, YYSTYPE value)
8189@{
8190 if (type == VAR)
8191 fprintf (file, "%s", value.tptr->name);
8192 else if (type == NUM)
8193 fprintf (file, "%d", value.val);
8194@}
8195@end smallexample
8196
8197@c ================================================= Invoking Bison
8198
8199@node Invocation
8200@chapter Invoking Bison
8201@cindex invoking Bison
8202@cindex Bison invocation
8203@cindex options for invoking Bison
8204
8205The usual way to invoke Bison is as follows:
8206
8207@example
8208bison @var{infile}
8209@end example
8210
8211Here @var{infile} is the grammar file name, which usually ends in
8212@samp{.y}. The parser file's name is made by replacing the @samp{.y}
8213with @samp{.tab.c} and removing any leading directory. Thus, the
8214@samp{bison foo.y} file name yields
8215@file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
8216@file{foo.tab.c}. It's also possible, in case you are writing
8217C++ code instead of C in your grammar file, to name it @file{foo.ypp}
8218or @file{foo.y++}. Then, the output files will take an extension like
8219the given one as input (respectively @file{foo.tab.cpp} and
8220@file{foo.tab.c++}).
8221This feature takes effect with all options that manipulate file names like
8222@samp{-o} or @samp{-d}.
8223
8224For example :
8225
8226@example
8227bison -d @var{infile.yxx}
8228@end example
8229@noindent
8230will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
8231
8232@example
8233bison -d -o @var{output.c++} @var{infile.y}
8234@end example
8235@noindent
8236will produce @file{output.c++} and @file{outfile.h++}.
8237
8238For compatibility with @acronym{POSIX}, the standard Bison
8239distribution also contains a shell script called @command{yacc} that
8240invokes Bison with the @option{-y} option.
8241
8242@menu
8243* Bison Options:: All the options described in detail,
8244 in alphabetical order by short options.
8245* Option Cross Key:: Alphabetical list of long options.
8246* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
8247@end menu
8248
8249@node Bison Options
8250@section Bison Options
8251
8252Bison supports both traditional single-letter options and mnemonic long
8253option names. Long option names are indicated with @samp{--} instead of
8254@samp{-}. Abbreviations for option names are allowed as long as they
8255are unique. When a long option takes an argument, like
8256@samp{--file-prefix}, connect the option name and the argument with
8257@samp{=}.
8258
8259Here is a list of options that can be used with Bison, alphabetized by
8260short option. It is followed by a cross key alphabetized by long
8261option.
8262
8263@c Please, keep this ordered as in `bison --help'.
8264@noindent
8265Operations modes:
8266@table @option
8267@item -h
8268@itemx --help
8269Print a summary of the command-line options to Bison and exit.
8270
8271@item -V
8272@itemx --version
8273Print the version number of Bison and exit.
8274
8275@item --print-localedir
8276Print the name of the directory containing locale-dependent data.
8277
8278@item --print-datadir
8279Print the name of the directory containing skeletons and XSLT.
8280
8281@item -y
8282@itemx --yacc
8283Act more like the traditional Yacc command. This can cause
8284different diagnostics to be generated, and may change behavior in
8285other minor ways. Most importantly, imitate Yacc's output
8286file name conventions, so that the parser output file is called
8287@file{y.tab.c}, and the other outputs are called @file{y.output} and
8288@file{y.tab.h}.
8289Also, if generating a deterministic parser in C, generate @code{#define}
8290statements in addition to an @code{enum} to associate token numbers with token
8291names.
8292Thus, the following shell script can substitute for Yacc, and the Bison
8293distribution contains such a script for compatibility with @acronym{POSIX}:
8294
8295@example
8296#! /bin/sh
8297bison -y "$@@"
8298@end example
8299
8300The @option{-y}/@option{--yacc} option is intended for use with
8301traditional Yacc grammars. If your grammar uses a Bison extension
8302like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
8303this option is specified.
8304
8305@item -W [@var{category}]
8306@itemx --warnings[=@var{category}]
8307Output warnings falling in @var{category}. @var{category} can be one
8308of:
8309@table @code
8310@item midrule-values
8311Warn about mid-rule values that are set but not used within any of the actions
8312of the parent rule.
8313For example, warn about unused @code{$2} in:
8314
8315@example
8316exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
8317@end example
8318
8319Also warn about mid-rule values that are used but not set.
8320For example, warn about unset @code{$$} in the mid-rule action in:
8321
8322@example
8323 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
8324@end example
8325
8326These warnings are not enabled by default since they sometimes prove to
8327be false alarms in existing grammars employing the Yacc constructs
8328@code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
8329
8330
8331@item yacc
8332Incompatibilities with @acronym{POSIX} Yacc.
8333
8334@item all
8335All the warnings.
8336@item none
8337Turn off all the warnings.
8338@item error
8339Treat warnings as errors.
8340@end table
8341
8342A category can be turned off by prefixing its name with @samp{no-}. For
8343instance, @option{-Wno-syntax} will hide the warnings about unused
8344variables.
8345@end table
8346
8347@noindent
8348Tuning the parser:
8349
8350@table @option
8351@item -t
8352@itemx --debug
8353In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
8354already defined, so that the debugging facilities are compiled.
8355@xref{Tracing, ,Tracing Your Parser}.
8356
8357@item -D @var{name}[=@var{value}]
8358@itemx --define=@var{name}[=@var{value}]
8359@itemx -F @var{name}[=@var{value}]
8360@itemx --force-define=@var{name}[=@var{value}]
8361Each of these is equivalent to @samp{%define @var{name} "@var{value}"}
8362(@pxref{Decl Summary, ,%define}) except that Bison processes multiple
8363definitions for the same @var{name} as follows:
8364
8365@itemize
8366@item
8367Bison quietly ignores all command-line definitions for @var{name} except
8368the last.
8369@item
8370If that command-line definition is specified by a @code{-D} or
8371@code{--define}, Bison reports an error for any @code{%define}
8372definition for @var{name}.
8373@item
8374If that command-line definition is specified by a @code{-F} or
8375@code{--force-define} instead, Bison quietly ignores all @code{%define}
8376definitions for @var{name}.
8377@item
8378Otherwise, Bison reports an error if there are multiple @code{%define}
8379definitions for @var{name}.
8380@end itemize
8381
8382You should avoid using @code{-F} and @code{--force-define} in your
8383makefiles unless you are confident that it is safe to quietly ignore any
8384conflicting @code{%define} that may be added to the grammar file.
8385
8386@item -L @var{language}
8387@itemx --language=@var{language}
8388Specify the programming language for the generated parser, as if
8389@code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
8390Summary}). Currently supported languages include C, C++, and Java.
8391@var{language} is case-insensitive.
8392
8393This option is experimental and its effect may be modified in future
8394releases.
8395
8396@item --locations
8397Pretend that @code{%locations} was specified. @xref{Decl Summary}.
8398
8399@item -p @var{prefix}
8400@itemx --name-prefix=@var{prefix}
8401Pretend that @code{%name-prefix "@var{prefix}"} was specified.
8402@xref{Decl Summary}.
8403
8404@item -l
8405@itemx --no-lines
8406Don't put any @code{#line} preprocessor commands in the parser file.
8407Ordinarily Bison puts them in the parser file so that the C compiler
8408and debuggers will associate errors with your source file, the
8409grammar file. This option causes them to associate errors with the
8410parser file, treating it as an independent source file in its own right.
8411
8412@item -S @var{file}
8413@itemx --skeleton=@var{file}
8414Specify the skeleton to use, similar to @code{%skeleton}
8415(@pxref{Decl Summary, , Bison Declaration Summary}).
8416
8417@c You probably don't need this option unless you are developing Bison.
8418@c You should use @option{--language} if you want to specify the skeleton for a
8419@c different language, because it is clearer and because it will always
8420@c choose the correct skeleton for non-deterministic or push parsers.
8421
8422If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
8423file in the Bison installation directory.
8424If it does, @var{file} is an absolute file name or a file name relative to the
8425current working directory.
8426This is similar to how most shells resolve commands.
8427
8428@item -k
8429@itemx --token-table
8430Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
8431@end table
8432
8433@noindent
8434Adjust the output:
8435
8436@table @option
8437@item --defines[=@var{file}]
8438Pretend that @code{%defines} was specified, i.e., write an extra output
8439file containing macro definitions for the token type names defined in
8440the grammar, as well as a few other declarations. @xref{Decl Summary}.
8441
8442@item -d
8443This is the same as @code{--defines} except @code{-d} does not accept a
8444@var{file} argument since POSIX Yacc requires that @code{-d} can be bundled
8445with other short options.
8446
8447@item -b @var{file-prefix}
8448@itemx --file-prefix=@var{prefix}
8449Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
8450for all Bison output file names. @xref{Decl Summary}.
8451
8452@item -r @var{things}
8453@itemx --report=@var{things}
8454Write an extra output file containing verbose description of the comma
8455separated list of @var{things} among:
8456
8457@table @code
8458@item state
8459Description of the grammar, conflicts (resolved and unresolved), and
8460parser's automaton.
8461
8462@item lookahead
8463Implies @code{state} and augments the description of the automaton with
8464each rule's lookahead set.
8465
8466@item itemset
8467Implies @code{state} and augments the description of the automaton with
8468the full set of items for each state, instead of its core only.
8469@end table
8470
8471@item --report-file=@var{file}
8472Specify the @var{file} for the verbose description.
8473
8474@item -v
8475@itemx --verbose
8476Pretend that @code{%verbose} was specified, i.e., write an extra output
8477file containing verbose descriptions of the grammar and
8478parser. @xref{Decl Summary}.
8479
8480@item -o @var{file}
8481@itemx --output=@var{file}
8482Specify the @var{file} for the parser file.
8483
8484The other output files' names are constructed from @var{file} as
8485described under the @samp{-v} and @samp{-d} options.
8486
8487@item -g [@var{file}]
8488@itemx --graph[=@var{file}]
8489Output a graphical representation of the parser's
8490automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
8491@uref{http://www.graphviz.org/doc/info/lang.html, @acronym{DOT}} format.
8492@code{@var{file}} is optional.
8493If omitted and the grammar file is @file{foo.y}, the output file will be
8494@file{foo.dot}.
8495
8496@item -x [@var{file}]
8497@itemx --xml[=@var{file}]
8498Output an XML report of the parser's automaton computed by Bison.
8499@code{@var{file}} is optional.
8500If omitted and the grammar file is @file{foo.y}, the output file will be
8501@file{foo.xml}.
8502(The current XML schema is experimental and may evolve.
8503More user feedback will help to stabilize it.)
8504@end table
8505
8506@node Option Cross Key
8507@section Option Cross Key
8508
8509Here is a list of options, alphabetized by long option, to help you find
8510the corresponding short option and directive.
8511
8512@multitable {@option{--force-define=@var{name}[=@var{value}]}} {@option{-F @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}}
8513@headitem Long Option @tab Short Option @tab Bison Directive
8514@include cross-options.texi
8515@end multitable
8516
8517@node Yacc Library
8518@section Yacc Library
8519
8520The Yacc library contains default implementations of the
8521@code{yyerror} and @code{main} functions. These default
8522implementations are normally not useful, but @acronym{POSIX} requires
8523them. To use the Yacc library, link your program with the
8524@option{-ly} option. Note that Bison's implementation of the Yacc
8525library is distributed under the terms of the @acronym{GNU} General
8526Public License (@pxref{Copying}).
8527
8528If you use the Yacc library's @code{yyerror} function, you should
8529declare @code{yyerror} as follows:
8530
8531@example
8532int yyerror (char const *);
8533@end example
8534
8535Bison ignores the @code{int} value returned by this @code{yyerror}.
8536If you use the Yacc library's @code{main} function, your
8537@code{yyparse} function should have the following type signature:
8538
8539@example
8540int yyparse (void);
8541@end example
8542
8543@c ================================================= C++ Bison
8544
8545@node Other Languages
8546@chapter Parsers Written In Other Languages
8547
8548@menu
8549* C++ Parsers:: The interface to generate C++ parser classes
8550* Java Parsers:: The interface to generate Java parser classes
8551@end menu
8552
8553@node C++ Parsers
8554@section C++ Parsers
8555
8556@menu
8557* C++ Bison Interface:: Asking for C++ parser generation
8558* C++ Semantic Values:: %union vs. C++
8559* C++ Location Values:: The position and location classes
8560* C++ Parser Interface:: Instantiating and running the parser
8561* C++ Scanner Interface:: Exchanges between yylex and parse
8562* A Complete C++ Example:: Demonstrating their use
8563@end menu
8564
8565@node C++ Bison Interface
8566@subsection C++ Bison Interface
8567@c - %skeleton "lalr1.cc"
8568@c - Always pure
8569@c - initial action
8570
8571The C++ deterministic parser is selected using the skeleton directive,
8572@samp{%skeleton "lalr1.c"}, or the synonymous command-line option
8573@option{--skeleton=lalr1.c}.
8574@xref{Decl Summary}.
8575
8576When run, @command{bison} will create several entities in the @samp{yy}
8577namespace.
8578@findex %define api.namespace
8579Use the @samp{%define api.namespace} directive to change the namespace
8580name, see
8581@ref{Decl Summary}.
8582The various classes are generated in the following files:
8583
8584@table @file
8585@item position.hh
8586@itemx location.hh
8587The definition of the classes @code{position} and @code{location},
8588used for location tracking when enabled. @xref{C++ Location Values}.
8589
8590@item stack.hh
8591An auxiliary class @code{stack} used by the parser.
8592
8593@item @var{file}.hh
8594@itemx @var{file}.cc
8595(Assuming the extension of the input file was @samp{.yy}.) The
8596declaration and implementation of the C++ parser class. The basename
8597and extension of these two files follow the same rules as with regular C
8598parsers (@pxref{Invocation}).
8599
8600The header is @emph{mandatory}; you must either pass
8601@option{-d}/@option{--defines} to @command{bison}, or use the
8602@samp{%defines} directive.
8603@end table
8604
8605All these files are documented using Doxygen; run @command{doxygen}
8606for a complete and accurate documentation.
8607
8608@node C++ Semantic Values
8609@subsection C++ Semantic Values
8610@c - No objects in unions
8611@c - YYSTYPE
8612@c - Printer and destructor
8613
8614Bison supports two different means to handle semantic values in C++. One is
8615alike the C interface, and relies on unions (@pxref{C++ Unions}). As C++
8616practitioners know, unions are inconvenient in C++, therefore another
8617approach is provided, based on variants (@pxref{C++ Variants}).
8618
8619@menu
8620* C++ Unions:: Semantic values cannot be objects
8621* C++ Variants:: Using objects as semantic values
8622@end menu
8623
8624@node C++ Unions
8625@subsubsection C++ Unions
8626
8627The @code{%union} directive works as for C, see @ref{Union Decl, ,The
8628Collection of Value Types}. In particular it produces a genuine
8629@code{union}, which have a few specific features in C++.
8630@itemize @minus
8631@item
8632The type @code{YYSTYPE} is defined but its use is discouraged: rather
8633you should refer to the parser's encapsulated type
8634@code{yy::parser::semantic_type}.
8635@item
8636Non POD (Plain Old Data) types cannot be used. C++ forbids any
8637instance of classes with constructors in unions: only @emph{pointers}
8638to such objects are allowed.
8639@end itemize
8640
8641Because objects have to be stored via pointers, memory is not
8642reclaimed automatically: using the @code{%destructor} directive is the
8643only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
8644Symbols}.
8645
8646@node C++ Variants
8647@subsubsection C++ Variants
8648
8649Starting with version 2.6, Bison provides a @emph{variant} based
8650implementation of semantic values for C++. This alleviates all the
8651limitations reported in the previous section, and in particular, object
8652types can be used without pointers.
8653
8654To enable variant-based semantic values, set @code{%define} variable
8655@code{variant} (@pxref{Decl Summary, , variant}). Once this defined,
8656@code{%union} is ignored, and instead of using the name of the fields of the
8657@code{%union} to ``type'' the symbols, use genuine types.
8658
8659For instance, instead of
8660
8661@example
8662%union
8663@{
8664 int ival;
8665 std::string* sval;
8666@}
8667%token <ival> NUMBER;
8668%token <sval> STRING;
8669@end example
8670
8671@noindent
8672write
8673
8674@example
8675%token <int> NUMBER;
8676%token <std::string> STRING;
8677@end example
8678
8679@code{STRING} is no longer a pointer, which should fairly simplify the user
8680actions in the grammar and in the scanner (in particular the memory
8681management).
8682
8683Since C++ features destructors, and since it is customary to specialize
8684@code{operator<<} to support uniform printing of values, variants also
8685typically simplify Bison printers and destructors.
8686
8687Variants are stricter than unions. When based on unions, you may play any
8688dirty game with @code{yylval}, say storing an @code{int}, reading a
8689@code{char*}, and then storing a @code{double} in it. This is no longer
8690possible with variants: they must be initialized, then assigned to, and
8691eventually, destroyed.
8692
8693@deftypemethod {semantic_type} {T&} build<T> ()
8694Initialize, but leave empty. Returns the address where the actual value may
8695be stored. Requires that the variant was not initialized yet.
8696@end deftypemethod
8697
8698@deftypemethod {semantic_type} {T&} build<T> (const T& @var{t})
8699Initialize, and copy-construct from @var{t}.
8700@end deftypemethod
8701
8702
8703@strong{Warning}: We do not use Boost.Variant, for two reasons. First, it
8704appeared unacceptable to require Boost on the user's machine (i.e., the
8705machine on which the generated parser will be compiled, not the machine on
8706which @command{bison} was run). Second, for each possible semantic value,
8707Boost.Variant not only stores the value, but also a tag specifying its
8708type. But the parser already ``knows'' the type of the semantic value, so
8709that would be duplicating the information.
8710
8711Therefore we developed light-weight variants whose type tag is external (so
8712they are really like @code{unions} for C++ actually). But our code is much
8713less mature that Boost.Variant. So there is a number of limitations in
8714(the current implementation of) variants:
8715@itemize
8716@item
8717Alignment must be enforced: values should be aligned in memory according to
8718the most demanding type. Computing the smallest alignment possible requires
8719meta-programming techniques that are not currently implemented in Bison, and
8720therefore, since, as far as we know, @code{double} is the most demanding
8721type on all platforms, alignments are enforced for @code{double} whatever
8722types are actually used. This may waste space in some cases.
8723
8724@item
8725Our implementation is not conforming with strict aliasing rules. Alias
8726analysis is a technique used in optimizing compilers to detect when two
8727pointers are disjoint (they cannot ``meet''). Our implementation breaks
8728some of the rules that G++ 4.4 uses in its alias analysis, so @emph{strict
8729alias analysis must be disabled}. Use the option
8730@option{-fno-strict-aliasing} to compile the generated parser.
8731
8732@item
8733There might be portability issues we are not aware of.
8734@end itemize
8735
8736As far as we know, these limitations @emph{can} be alleviated. All it takes
8737is some time and/or some talented C++ hacker willing to contribute to Bison.
8738
8739@node C++ Location Values
8740@subsection C++ Location Values
8741@c - %locations
8742@c - class Position
8743@c - class Location
8744@c - %define filename_type "const symbol::Symbol"
8745
8746When the directive @code{%locations} is used, the C++ parser supports
8747location tracking, see @ref{Locations, , Locations Overview}. Two
8748auxiliary classes define a @code{position}, a single point in a file,
8749and a @code{location}, a range composed of a pair of
8750@code{position}s (possibly spanning several files).
8751
8752@deftypemethod {position} {std::string*} file
8753The name of the file. It will always be handled as a pointer, the
8754parser will never duplicate nor deallocate it. As an experimental
8755feature you may change it to @samp{@var{type}*} using @samp{%define
8756filename_type "@var{type}"}.
8757@end deftypemethod
8758
8759@deftypemethod {position} {unsigned int} line
8760The line, starting at 1.
8761@end deftypemethod
8762
8763@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
8764Advance by @var{height} lines, resetting the column number.
8765@end deftypemethod
8766
8767@deftypemethod {position} {unsigned int} column
8768The column, starting at 0.
8769@end deftypemethod
8770
8771@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
8772Advance by @var{width} columns, without changing the line number.
8773@end deftypemethod
8774
8775@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
8776@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
8777@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
8778@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
8779Various forms of syntactic sugar for @code{columns}.
8780@end deftypemethod
8781
8782@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
8783Report @var{p} on @var{o} like this:
8784@samp{@var{file}:@var{line}.@var{column}}, or
8785@samp{@var{line}.@var{column}} if @var{file} is null.
8786@end deftypemethod
8787
8788@deftypemethod {location} {position} begin
8789@deftypemethodx {location} {position} end
8790The first, inclusive, position of the range, and the first beyond.
8791@end deftypemethod
8792
8793@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
8794@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
8795Advance the @code{end} position.
8796@end deftypemethod
8797
8798@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
8799@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
8800@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
8801Various forms of syntactic sugar.
8802@end deftypemethod
8803
8804@deftypemethod {location} {void} step ()
8805Move @code{begin} onto @code{end}.
8806@end deftypemethod
8807
8808
8809@node C++ Parser Interface
8810@subsection C++ Parser Interface
8811@c - define parser_class_name
8812@c - Ctor
8813@c - parse, error, set_debug_level, debug_level, set_debug_stream,
8814@c debug_stream.
8815@c - Reporting errors
8816
8817The output files @file{@var{output}.hh} and @file{@var{output}.cc}
8818declare and define the parser class in the namespace @code{yy}. The
8819class name defaults to @code{parser}, but may be changed using
8820@samp{%define parser_class_name "@var{name}"}. The interface of
8821this class is detailed below. It can be extended using the
8822@code{%parse-param} feature: its semantics is slightly changed since
8823it describes an additional member of the parser class, and an
8824additional argument for its constructor.
8825
8826@defcv {Type} {parser} {semantic_type}
8827@defcvx {Type} {parser} {location_type}
8828The types for semantic values and locations (if enabled).
8829@end defcv
8830
8831@defcv {Type} {parser} {syntax_error}
8832This class derives from @code{std::runtime_error}. Throw instances of it
8833from user actions to raise parse errors. This is equivalent with first
8834invoking @code{error} to report the location and message of the syntax
8835error, and then to invoke @code{YYERROR} to enter the error-recovery mode.
8836But contrary to @code{YYERROR} which can only be invoked from user actions
8837(i.e., written in the action itself), the exception can be thrown from
8838function invoked from the user action.
8839@end defcv
8840
8841@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
8842Build a new parser object. There are no arguments by default, unless
8843@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
8844@end deftypemethod
8845
8846@deftypemethod {syntax_error} {} syntax_error (const location_type& @var{l}, const std::string& @var{m})
8847@deftypemethodx {syntax_error} {} syntax_error (const std::string& @var{m})
8848Instantiate a syntax-error exception.
8849@end deftypemethod
8850
8851@deftypemethod {parser} {int} parse ()
8852Run the syntactic analysis, and return 0 on success, 1 otherwise.
8853@end deftypemethod
8854
8855@deftypemethod {parser} {std::ostream&} debug_stream ()
8856@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
8857Get or set the stream used for tracing the parsing. It defaults to
8858@code{std::cerr}.
8859@end deftypemethod
8860
8861@deftypemethod {parser} {debug_level_type} debug_level ()
8862@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
8863Get or set the tracing level. Currently its value is either 0, no trace,
8864or nonzero, full tracing.
8865@end deftypemethod
8866
8867@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
8868@deftypemethodx {parser} {void} error (const std::string& @var{m})
8869The definition for this member function must be supplied by the user:
8870the parser uses it to report a parser error occurring at @var{l},
8871described by @var{m}. If location tracking is not enabled, the second
8872signature is used.
8873@end deftypemethod
8874
8875
8876@node C++ Scanner Interface
8877@subsection C++ Scanner Interface
8878@c - prefix for yylex.
8879@c - Pure interface to yylex
8880@c - %lex-param
8881
8882The parser invokes the scanner by calling @code{yylex}. Contrary to C
8883parsers, C++ parsers are always pure: there is no point in using the
8884@samp{%define api.pure} directive. The actual interface with @code{yylex}
8885depends whether you use unions, or variants.
8886
8887@menu
8888* Split Symbols:: Passing symbols as two/three components
8889* Complete Symbols:: Making symbols a whole
8890@end menu
8891
8892@node Split Symbols
8893@subsubsection Split Symbols
8894
8895Therefore the interface is as follows.
8896
8897@deftypemethod {parser} {int} yylex (semantic_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...)
8898@deftypemethodx {parser} {int} yylex (semantic_type& @var{yylval}, @var{type1} @var{arg1}, ...)
8899Return the next token. Its type is the return value, its semantic value and
8900location (if enabled) being @var{yylval} and @var{yylloc}. Invocations of
8901@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
8902@end deftypemethod
8903
8904Note that when using variants, the interface for @code{yylex} is the same,
8905but @code{yylval} is handled differently.
8906
8907Regular union-based code in Lex scanner typically look like:
8908
8909@example
8910[0-9]+ @{
8911 yylval.ival = text_to_int (yytext);
8912 return yy::parser::INTEGER;
8913 @}
8914[a-z]+ @{
8915 yylval.sval = new std::string (yytext);
8916 return yy::parser::IDENTIFIER;
8917 @}
8918@end example
8919
8920Using variants, @code{yylval} is already constructed, but it is not
8921initialized. So the code would look like:
8922
8923@example
8924[0-9]+ @{
8925 yylval.build<int>() = text_to_int (yytext);
8926 return yy::parser::INTEGER;
8927 @}
8928[a-z]+ @{
8929 yylval.build<std::string> = yytext;
8930 return yy::parser::IDENTIFIER;
8931 @}
8932@end example
8933
8934@noindent
8935or
8936
8937@example
8938[0-9]+ @{
8939 yylval.build(text_to_int (yytext));
8940 return yy::parser::INTEGER;
8941 @}
8942[a-z]+ @{
8943 yylval.build(yytext);
8944 return yy::parser::IDENTIFIER;
8945 @}
8946@end example
8947
8948
8949@node Complete Symbols
8950@subsubsection Complete Symbols
8951
8952If you specified both @code{%define variant} and @code{%define lex_symbol},
8953the @code{parser} class also defines the class @code{parser::symbol_type}
8954which defines a @emph{complete} symbol, aggregating its type (i.e., the
8955traditional value returned by @code{yylex}), its semantic value (i.e., the
8956value passed in @code{yylval}, and possibly its location (@code{yylloc}).
8957
8958@deftypemethod {symbol_type} {} symbol_type (token_type @var{type}, const semantic_type& @var{value}, const location_type& @var{location})
8959Build a complete terminal symbol which token type is @var{type}, and which
8960semantic value is @var{value}. If location tracking is enabled, also pass
8961the @var{location}.
8962@end deftypemethod
8963
8964This interface is low-level and should not be used for two reasons. First,
8965it is inconvenient, as you still have to build the semantic value, which is
8966a variant, and second, because consistency is not enforced: as with unions,
8967it is still possible to give an integer as semantic value for a string.
8968
8969So for each token type, Bison generates named constructors as follows.
8970
8971@deftypemethod {symbol_type} {} make_@var{token} (const @var{value_type}& @var{value}, const location_type& @var{location})
8972@deftypemethodx {symbol_type} {} make_@var{token} (const location_type& @var{location})
8973Build a complete terminal symbol for the token type @var{token} (not
8974including the @code{api.tokens.prefix}) whose possible semantic value is
8975@var{value} of adequate @var{value_type}. If location tracking is enabled,
8976also pass the @var{location}.
8977@end deftypemethod
8978
8979For instance, given the following declarations:
8980
8981@example
8982%define api.tokens.prefix "TOK_"
8983%token <std::string> IDENTIFIER;
8984%token <int> INTEGER;
8985%token COLON;
8986@end example
8987
8988@noindent
8989Bison generates the following functions:
8990
8991@example
8992symbol_type make_IDENTIFIER(const std::string& v,
8993 const location_type& l);
8994symbol_type make_INTEGER(const int& v,
8995 const location_type& loc);
8996symbol_type make_COLON(const location_type& loc);
8997@end example
8998
8999@noindent
9000which should be used in a Lex-scanner as follows.
9001
9002@example
9003[0-9]+ return yy::parser::make_INTEGER(text_to_int (yytext), loc);
9004[a-z]+ return yy::parser::make_IDENTIFIER(yytext, loc);
9005":" return yy::parser::make_COLON(loc);
9006@end example
9007
9008Tokens that do not have an identifier are not accessible: you cannot simply
9009use characters such as @code{':'}, they must be declared with @code{%token}.
9010
9011@node A Complete C++ Example
9012@subsection A Complete C++ Example
9013
9014This section demonstrates the use of a C++ parser with a simple but
9015complete example. This example should be available on your system,
9016ready to compile, in the directory @dfn{.../bison/examples/calc++}. It
9017focuses on the use of Bison, therefore the design of the various C++
9018classes is very naive: no accessors, no encapsulation of members etc.
9019We will use a Lex scanner, and more precisely, a Flex scanner, to
9020demonstrate the various interactions. A hand-written scanner is
9021actually easier to interface with.
9022
9023@menu
9024* Calc++ --- C++ Calculator:: The specifications
9025* Calc++ Parsing Driver:: An active parsing context
9026* Calc++ Parser:: A parser class
9027* Calc++ Scanner:: A pure C++ Flex scanner
9028* Calc++ Top Level:: Conducting the band
9029@end menu
9030
9031@node Calc++ --- C++ Calculator
9032@subsubsection Calc++ --- C++ Calculator
9033
9034Of course the grammar is dedicated to arithmetics, a single
9035expression, possibly preceded by variable assignments. An
9036environment containing possibly predefined variables such as
9037@code{one} and @code{two}, is exchanged with the parser. An example
9038of valid input follows.
9039
9040@example
9041three := 3
9042seven := one + two * three
9043seven * seven
9044@end example
9045
9046@node Calc++ Parsing Driver
9047@subsubsection Calc++ Parsing Driver
9048@c - An env
9049@c - A place to store error messages
9050@c - A place for the result
9051
9052To support a pure interface with the parser (and the scanner) the
9053technique of the ``parsing context'' is convenient: a structure
9054containing all the data to exchange. Since, in addition to simply
9055launch the parsing, there are several auxiliary tasks to execute (open
9056the file for parsing, instantiate the parser etc.), we recommend
9057transforming the simple parsing context structure into a fully blown
9058@dfn{parsing driver} class.
9059
9060The declaration of this driver class, @file{calc++-driver.hh}, is as
9061follows. The first part includes the CPP guard and imports the
9062required standard library components, and the declaration of the parser
9063class.
9064
9065@comment file: calc++-driver.hh
9066@example
9067#ifndef CALCXX_DRIVER_HH
9068# define CALCXX_DRIVER_HH
9069# include <string>
9070# include <map>
9071# include "calc++-parser.hh"
9072@end example
9073
9074
9075@noindent
9076Then comes the declaration of the scanning function. Flex expects
9077the signature of @code{yylex} to be defined in the macro
9078@code{YY_DECL}, and the C++ parser expects it to be declared. We can
9079factor both as follows.
9080
9081@comment file: calc++-driver.hh
9082@example
9083// Tell Flex the lexer's prototype ...
9084# define YY_DECL \
9085 yy::calcxx_parser::symbol_type yylex (calcxx_driver& driver)
9086// ... and declare it for the parser's sake.
9087YY_DECL;
9088@end example
9089
9090@noindent
9091The @code{calcxx_driver} class is then declared with its most obvious
9092members.
9093
9094@comment file: calc++-driver.hh
9095@example
9096// Conducting the whole scanning and parsing of Calc++.
9097class calcxx_driver
9098@{
9099public:
9100 calcxx_driver ();
9101 virtual ~calcxx_driver ();
9102
9103 std::map<std::string, int> variables;
9104
9105 int result;
9106@end example
9107
9108@noindent
9109To encapsulate the coordination with the Flex scanner, it is useful to have
9110member functions to open and close the scanning phase.
9111
9112@comment file: calc++-driver.hh
9113@example
9114 // Handling the scanner.
9115 void scan_begin ();
9116 void scan_end ();
9117 bool trace_scanning;
9118@end example
9119
9120@noindent
9121Similarly for the parser itself.
9122
9123@comment file: calc++-driver.hh
9124@example
9125 // Run the parser on file F.
9126 // Return 0 on success.
9127 int parse (const std::string& f);
9128 // The name of the file being parsed.
9129 // Used later to pass the file name to the location tracker.
9130 std::string file;
9131 // Whether parser traces should be generated.
9132 bool trace_parsing;
9133@end example
9134
9135@noindent
9136To demonstrate pure handling of parse errors, instead of simply
9137dumping them on the standard error output, we will pass them to the
9138compiler driver using the following two member functions. Finally, we
9139close the class declaration and CPP guard.
9140
9141@comment file: calc++-driver.hh
9142@example
9143 // Error handling.
9144 void error (const yy::location& l, const std::string& m);
9145 void error (const std::string& m);
9146@};
9147#endif // ! CALCXX_DRIVER_HH
9148@end example
9149
9150The implementation of the driver is straightforward. The @code{parse}
9151member function deserves some attention. The @code{error} functions
9152are simple stubs, they should actually register the located error
9153messages and set error state.
9154
9155@comment file: calc++-driver.cc
9156@example
9157#include "calc++-driver.hh"
9158#include "calc++-parser.hh"
9159
9160calcxx_driver::calcxx_driver ()
9161 : trace_scanning (false), trace_parsing (false)
9162@{
9163 variables["one"] = 1;
9164 variables["two"] = 2;
9165@}
9166
9167calcxx_driver::~calcxx_driver ()
9168@{
9169@}
9170
9171int
9172calcxx_driver::parse (const std::string &f)
9173@{
9174 file = f;
9175 scan_begin ();
9176 yy::calcxx_parser parser (*this);
9177 parser.set_debug_level (trace_parsing);
9178 int res = parser.parse ();
9179 scan_end ();
9180 return res;
9181@}
9182
9183void
9184calcxx_driver::error (const yy::location& l, const std::string& m)
9185@{
9186 std::cerr << l << ": " << m << std::endl;
9187@}
9188
9189void
9190calcxx_driver::error (const std::string& m)
9191@{
9192 std::cerr << m << std::endl;
9193@}
9194@end example
9195
9196@node Calc++ Parser
9197@subsubsection Calc++ Parser
9198
9199The parser definition file @file{calc++-parser.yy} starts by asking for
9200the C++ deterministic parser skeleton, the creation of the parser header
9201file, and specifies the name of the parser class.
9202Because the C++ skeleton changed several times, it is safer to require
9203the version you designed the grammar for.
9204
9205@comment file: calc++-parser.yy
9206@example
9207%skeleton "lalr1.cc" /* -*- C++ -*- */
9208%require "@value{VERSION}"
9209%defines
9210%define parser_class_name "calcxx_parser"
9211@end example
9212
9213@noindent
9214@findex %define variant
9215@findex %define lex_symbol
9216This example will use genuine C++ objects as semantic values, therefore, we
9217require the variant-based interface. To make sure we properly use it, we
9218enable assertions. To fully benefit from type-safety and more natural
9219definition of ``symbol'', we enable @code{lex_symbol}.
9220
9221@comment file: calc++-parser.yy
9222@example
9223%define variant
9224%define parse.assert
9225%define lex_symbol
9226@end example
9227
9228@noindent
9229@findex %code requires
9230Then come the declarations/inclusions needed by the semantic values.
9231Because the parser uses the parsing driver and reciprocally, both would like
9232to include the header of the other, which is, of course, insane. This
9233mutual dependency will be broken using forward declarations. Because the
9234driver's header needs detailed knowledge about the parser class (in
9235particular its inner types), it is the parser's header which will use a
9236forward declaration of the driver. @xref{Decl Summary, ,%code}.
9237
9238@comment file: calc++-parser.yy
9239@example
9240%code requires
9241@{
9242# include <string>
9243class calcxx_driver;
9244@}
9245@end example
9246
9247@noindent
9248The driver is passed by reference to the parser and to the scanner.
9249This provides a simple but effective pure interface, not relying on
9250global variables.
9251
9252@comment file: calc++-parser.yy
9253@example
9254// The parsing context.
9255%param @{ calcxx_driver& driver @}
9256@end example
9257
9258@noindent
9259Then we request location tracking, and initialize the
9260first location's file name. Afterward new locations are computed
9261relatively to the previous locations: the file name will be
9262propagated.
9263
9264@comment file: calc++-parser.yy
9265@example
9266%locations
9267%initial-action
9268@{
9269 // Initialize the initial location.
9270 @@$.begin.filename = @@$.end.filename = &driver.file;
9271@};
9272@end example
9273
9274@noindent
9275Use the following two directives to enable parser tracing and verbose
9276error messages.
9277
9278@comment file: calc++-parser.yy
9279@example
9280%define parse.trace
9281%define parse.error verbose
9282@end example
9283
9284@noindent
9285@findex %code
9286The code between @samp{%code @{} and @samp{@}} is output in the
9287@file{*.cc} file; it needs detailed knowledge about the driver.
9288
9289@comment file: calc++-parser.yy
9290@example
9291%code
9292@{
9293# include "calc++-driver.hh"
9294@}
9295@end example
9296
9297
9298@noindent
9299The token numbered as 0 corresponds to end of file; the following line
9300allows for nicer error messages referring to ``end of file'' instead of
9301``$end''. Similarly user friendly names are provided for each symbol.
9302To avoid name clashes in the generated files (@pxref{Calc++ Scanner}),
9303prefix tokens with @code{TOK_} (@pxref{Decl Summary,, api.tokens.prefix}).
9304
9305@comment file: calc++-parser.yy
9306@example
9307%define api.tokens.prefix "TOK_"
9308%token
9309 END 0 "end of file"
9310 ASSIGN ":="
9311 MINUS "-"
9312 PLUS "+"
9313 STAR "*"
9314 SLASH "/"
9315 LPAREN "("
9316 RPAREN ")"
9317;
9318@end example
9319
9320@noindent
9321Since we use variant-based semantic values, @code{%union} is not used, and
9322both @code{%type} and @code{%token} expect genuine types, as opposed to type
9323tags.
9324
9325@comment file: calc++-parser.yy
9326@example
9327%token <std::string> IDENTIFIER "identifier"
9328%token <int> NUMBER "number"
9329%type <int> exp
9330@end example
9331
9332@noindent
9333No @code{%destructor} is needed to enable memory deallocation during error
9334recovery; the memory, for strings for instance, will be reclaimed by the
9335regular destructors. All the values are printed using their
9336@code{operator<<}.
9337
9338@c FIXME: Document %printer, and mention that it takes a braced-code operand.
9339@comment file: calc++-parser.yy
9340@example
9341%printer @{ debug_stream () << $$; @} <*>;
9342@end example
9343
9344@noindent
9345The grammar itself is straightforward (@pxref{Location Tracking Calc, ,
9346Location Tracking Calculator: @code{ltcalc}}).
9347
9348@comment file: calc++-parser.yy
9349@example
9350%%
9351%start unit;
9352unit: assignments exp @{ driver.result = $2; @};
9353
9354assignments:
9355 assignments assignment @{@}
9356| /* Nothing. */ @{@};
9357
9358assignment:
9359 "identifier" ":=" exp @{ driver.variables[$1] = $3; @};
9360
9361%left "+" "-";
9362%left "*" "/";
9363exp:
9364 exp "+" exp @{ $$ = $1 + $3; @}
9365| exp "-" exp @{ $$ = $1 - $3; @}
9366| exp "*" exp @{ $$ = $1 * $3; @}
9367| exp "/" exp @{ $$ = $1 / $3; @}
9368| "(" exp ")" @{ std::swap ($$, $2); @}
9369| "identifier" @{ $$ = driver.variables[$1]; @}
9370| "number" @{ std::swap ($$, $1); @};
9371%%
9372@end example
9373
9374@noindent
9375Finally the @code{error} member function registers the errors to the
9376driver.
9377
9378@comment file: calc++-parser.yy
9379@example
9380void
9381yy::calcxx_parser::error (const location_type& l,
9382 const std::string& m)
9383@{
9384 driver.error (l, m);
9385@}
9386@end example
9387
9388@node Calc++ Scanner
9389@subsubsection Calc++ Scanner
9390
9391The Flex scanner first includes the driver declaration, then the
9392parser's to get the set of defined tokens.
9393
9394@comment file: calc++-scanner.ll
9395@example
9396%@{ /* -*- C++ -*- */
9397# include <cerrno>
9398# include <climits>
9399# include <cstdlib>
9400# include <string>
9401# include "calc++-driver.hh"
9402# include "calc++-parser.hh"
9403
9404// Work around an incompatibility in flex (at least versions
9405// 2.5.31 through 2.5.33): it generates code that does
9406// not conform to C89. See Debian bug 333231
9407// <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>.
9408# undef yywrap
9409# define yywrap() 1
9410
9411// The location of the current token.
9412static yy::location loc;
9413%@}
9414@end example
9415
9416@noindent
9417Because there is no @code{#include}-like feature we don't need
9418@code{yywrap}, we don't need @code{unput} either, and we parse an
9419actual file, this is not an interactive session with the user.
9420Finally, we enable scanner tracing.
9421
9422@comment file: calc++-scanner.ll
9423@example
9424%option noyywrap nounput batch debug
9425@end example
9426
9427@noindent
9428Abbreviations allow for more readable rules.
9429
9430@comment file: calc++-scanner.ll
9431@example
9432id [a-zA-Z][a-zA-Z_0-9]*
9433int [0-9]+
9434blank [ \t]
9435@end example
9436
9437@noindent
9438The following paragraph suffices to track locations accurately. Each
9439time @code{yylex} is invoked, the begin position is moved onto the end
9440position. Then when a pattern is matched, its width is added to the end
9441column. When matching ends of lines, the end
9442cursor is adjusted, and each time blanks are matched, the begin cursor
9443is moved onto the end cursor to effectively ignore the blanks
9444preceding tokens. Comments would be treated equally.
9445
9446@comment file: calc++-scanner.ll
9447@example
9448%@{
9449 // Code run each time a pattern is matched.
9450 # define YY_USER_ACTION loc.columns (yyleng);
9451%@}
9452%%
9453%@{
9454 // Code run each time yylex is called.
9455 loc.step ();
9456%@}
9457@{blank@}+ loc.step ();
9458[\n]+ loc.lines (yyleng); loc.step ();
9459@end example
9460
9461@noindent
9462The rules are simple. The driver is used to report errors.
9463
9464@comment file: calc++-scanner.ll
9465@example
9466"-" return yy::calcxx_parser::make_MINUS(loc);
9467"+" return yy::calcxx_parser::make_PLUS(loc);
9468"*" return yy::calcxx_parser::make_STAR(loc);
9469"/" return yy::calcxx_parser::make_SLASH(loc);
9470"(" return yy::calcxx_parser::make_LPAREN(loc);
9471")" return yy::calcxx_parser::make_RPAREN(loc);
9472":=" return yy::calcxx_parser::make_ASSIGN(loc);
9473
9474@{int@} @{
9475 errno = 0;
9476 long n = strtol (yytext, NULL, 10);
9477 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
9478 driver.error (loc, "integer is out of range");
9479 return yy::calcxx_parser::make_NUMBER(n, loc);
9480@}
9481@{id@} return yy::calcxx_parser::make_IDENTIFIER(yytext, loc);
9482. driver.error (loc, "invalid character");
9483<<EOF>> return yy::calcxx_parser::make_END(loc);
9484%%
9485@end example
9486
9487@noindent
9488Finally, because the scanner-related driver's member-functions depend
9489on the scanner's data, it is simpler to implement them in this file.
9490
9491@comment file: calc++-scanner.ll
9492@example
9493void
9494calcxx_driver::scan_begin ()
9495@{
9496 yy_flex_debug = trace_scanning;
9497 if (file == "-")
9498 yyin = stdin;
9499 else if (!(yyin = fopen (file.c_str (), "r")))
9500 @{
9501 error (std::string ("cannot open ") + file + ": " + strerror(errno));
9502 exit (1);
9503 @}
9504@}
9505
9506void
9507calcxx_driver::scan_end ()
9508@{
9509 fclose (yyin);
9510@}
9511@end example
9512
9513@node Calc++ Top Level
9514@subsubsection Calc++ Top Level
9515
9516The top level file, @file{calc++.cc}, poses no problem.
9517
9518@comment file: calc++.cc
9519@example
9520#include <iostream>
9521#include "calc++-driver.hh"
9522
9523int
9524main (int argc, char *argv[])
9525@{
9526 int res = 0;
9527 calcxx_driver driver;
9528 for (++argv; argv[0]; ++argv)
9529 if (*argv == std::string ("-p"))
9530 driver.trace_parsing = true;
9531 else if (*argv == std::string ("-s"))
9532 driver.trace_scanning = true;
9533 else if (!driver.parse (*argv))
9534 std::cout << driver.result << std::endl;
9535 else
9536 res = 1;
9537 return res;
9538@}
9539@end example
9540
9541@node Java Parsers
9542@section Java Parsers
9543
9544@menu
9545* Java Bison Interface:: Asking for Java parser generation
9546* Java Semantic Values:: %type and %token vs. Java
9547* Java Location Values:: The position and location classes
9548* Java Parser Interface:: Instantiating and running the parser
9549* Java Scanner Interface:: Specifying the scanner for the parser
9550* Java Action Features:: Special features for use in actions
9551* Java Differences:: Differences between C/C++ and Java Grammars
9552* Java Declarations Summary:: List of Bison declarations used with Java
9553@end menu
9554
9555@node Java Bison Interface
9556@subsection Java Bison Interface
9557@c - %language "Java"
9558
9559(The current Java interface is experimental and may evolve.
9560More user feedback will help to stabilize it.)
9561
9562The Java parser skeletons are selected using the @code{%language "Java"}
9563directive or the @option{-L java}/@option{--language=java} option.
9564
9565@c FIXME: Documented bug.
9566When generating a Java parser, @code{bison @var{basename}.y} will create
9567a single Java source file named @file{@var{basename}.java}. Using an
9568input file without a @file{.y} suffix is currently broken. The basename
9569of the output file can be changed by the @code{%file-prefix} directive
9570or the @option{-p}/@option{--name-prefix} option. The entire output file
9571name can be changed by the @code{%output} directive or the
9572@option{-o}/@option{--output} option. The output file contains a single
9573class for the parser.
9574
9575You can create documentation for generated parsers using Javadoc.
9576
9577Contrary to C parsers, Java parsers do not use global variables; the
9578state of the parser is always local to an instance of the parser class.
9579Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
9580and @samp{%define api.pure} directives does not do anything when used in
9581Java.
9582
9583Push parsers are currently unsupported in Java and @code{%define
9584api.push-pull} have no effect.
9585
9586@acronym{GLR} parsers are currently unsupported in Java. Do not use the
9587@code{glr-parser} directive.
9588
9589No header file can be generated for Java parsers. Do not use the
9590@code{%defines} directive or the @option{-d}/@option{--defines} options.
9591
9592@c FIXME: Possible code change.
9593Currently, support for tracing is always compiled
9594in. Thus the @samp{%define parse.trace} and @samp{%token-table}
9595directives and the
9596@option{-t}/@option{--debug} and @option{-k}/@option{--token-table}
9597options have no effect. This may change in the future to eliminate
9598unused code in the generated parser, so use @samp{%define parse.trace}
9599explicitly
9600if needed. Also, in the future the
9601@code{%token-table} directive might enable a public interface to
9602access the token names and codes.
9603
9604Getting a ``code too large'' error from the Java compiler means the code
9605hit the 64KB bytecode per method limitation of the Java class file.
9606Try reducing the amount of code in actions and static initializers;
9607otherwise, report a bug so that the parser skeleton will be improved.
9608
9609
9610@node Java Semantic Values
9611@subsection Java Semantic Values
9612@c - No %union, specify type in %type/%token.
9613@c - YYSTYPE
9614@c - Printer and destructor
9615
9616There is no @code{%union} directive in Java parsers. Instead, the
9617semantic values' types (class names) should be specified in the
9618@code{%type} or @code{%token} directive:
9619
9620@example
9621%type <Expression> expr assignment_expr term factor
9622%type <Integer> number
9623@end example
9624
9625By default, the semantic stack is declared to have @code{Object} members,
9626which means that the class types you specify can be of any class.
9627To improve the type safety of the parser, you can declare the common
9628superclass of all the semantic values using the @samp{%define stype}
9629directive. For example, after the following declaration:
9630
9631@example
9632%define stype "ASTNode"
9633@end example
9634
9635@noindent
9636any @code{%type} or @code{%token} specifying a semantic type which
9637is not a subclass of ASTNode, will cause a compile-time error.
9638
9639@c FIXME: Documented bug.
9640Types used in the directives may be qualified with a package name.
9641Primitive data types are accepted for Java version 1.5 or later. Note
9642that in this case the autoboxing feature of Java 1.5 will be used.
9643Generic types may not be used; this is due to a limitation in the
9644implementation of Bison, and may change in future releases.
9645
9646Java parsers do not support @code{%destructor}, since the language
9647adopts garbage collection. The parser will try to hold references
9648to semantic values for as little time as needed.
9649
9650Java parsers do not support @code{%printer}, as @code{toString()}
9651can be used to print the semantic values. This however may change
9652(in a backwards-compatible way) in future versions of Bison.
9653
9654
9655@node Java Location Values
9656@subsection Java Location Values
9657@c - %locations
9658@c - class Position
9659@c - class Location
9660
9661When the directive @code{%locations} is used, the Java parser
9662supports location tracking, see @ref{Locations, , Locations Overview}.
9663An auxiliary user-defined class defines a @dfn{position}, a single point
9664in a file; Bison itself defines a class representing a @dfn{location},
9665a range composed of a pair of positions (possibly spanning several
9666files). The location class is an inner class of the parser; the name
9667is @code{Location} by default, and may also be renamed using
9668@samp{%define location_type "@var{class-name}"}.
9669
9670The location class treats the position as a completely opaque value.
9671By default, the class name is @code{Position}, but this can be changed
9672with @samp{%define position_type "@var{class-name}"}. This class must
9673be supplied by the user.
9674
9675
9676@deftypeivar {Location} {Position} begin
9677@deftypeivarx {Location} {Position} end
9678The first, inclusive, position of the range, and the first beyond.
9679@end deftypeivar
9680
9681@deftypeop {Constructor} {Location} {} Location (Position @var{loc})
9682Create a @code{Location} denoting an empty range located at a given point.
9683@end deftypeop
9684
9685@deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end})
9686Create a @code{Location} from the endpoints of the range.
9687@end deftypeop
9688
9689@deftypemethod {Location} {String} toString ()
9690Prints the range represented by the location. For this to work
9691properly, the position class should override the @code{equals} and
9692@code{toString} methods appropriately.
9693@end deftypemethod
9694
9695
9696@node Java Parser Interface
9697@subsection Java Parser Interface
9698@c - define parser_class_name
9699@c - Ctor
9700@c - parse, error, set_debug_level, debug_level, set_debug_stream,
9701@c debug_stream.
9702@c - Reporting errors
9703
9704The name of the generated parser class defaults to @code{YYParser}. The
9705@code{YY} prefix may be changed using the @code{%name-prefix} directive
9706or the @option{-p}/@option{--name-prefix} option. Alternatively, use
9707@samp{%define parser_class_name "@var{name}"} to give a custom name to
9708the class. The interface of this class is detailed below.
9709
9710By default, the parser class has package visibility. A declaration
9711@samp{%define public} will change to public visibility. Remember that,
9712according to the Java language specification, the name of the @file{.java}
9713file should match the name of the class in this case. Similarly, you can
9714use @code{abstract}, @code{final} and @code{strictfp} with the
9715@code{%define} declaration to add other modifiers to the parser class.
9716A single @samp{%define annotations "@var{annotations}"} directive can
9717be used to add any number of annotations to the parser class.
9718
9719The Java package name of the parser class can be specified using the
9720@samp{%define package} directive. The superclass and the implemented
9721interfaces of the parser class can be specified with the @code{%define
9722extends} and @samp{%define implements} directives.
9723
9724The parser class defines an inner class, @code{Location}, that is used
9725for location tracking (see @ref{Java Location Values}), and a inner
9726interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
9727these inner class/interface, and the members described in the interface
9728below, all the other members and fields are preceded with a @code{yy} or
9729@code{YY} prefix to avoid clashes with user code.
9730
9731The parser class can be extended using the @code{%parse-param}
9732directive. Each occurrence of the directive will add a @code{protected
9733final} field to the parser class, and an argument to its constructor,
9734which initialize them automatically.
9735
9736@deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
9737Build a new parser object with embedded @code{%code lexer}. There are
9738no parameters, unless @code{%param}s and/or @code{%parse-param}s and/or
9739@code{%lex-param}s are used.
9740
9741Use @code{%code init} for code added to the start of the constructor
9742body. This is especially useful to initialize superclasses. Use
9743@samp{%define init_throws} to specify any uncaught exceptions.
9744@end deftypeop
9745
9746@deftypeop {Constructor} {YYParser} {} YYParser (Lexer @var{lexer}, @var{parse_param}, @dots{})
9747Build a new parser object using the specified scanner. There are no
9748additional parameters unless @code{%param}s and/or @code{%parse-param}s are
9749used.
9750
9751If the scanner is defined by @code{%code lexer}, this constructor is
9752declared @code{protected} and is called automatically with a scanner
9753created with the correct @code{%param}s and/or @code{%lex-param}s.
9754
9755Use @code{%code init} for code added to the start of the constructor
9756body. This is especially useful to initialize superclasses. Use
9757@samp{%define init_throws} to specify any uncatch exceptions.
9758@end deftypeop
9759
9760@deftypemethod {YYParser} {boolean} parse ()
9761Run the syntactic analysis, and return @code{true} on success,
9762@code{false} otherwise.
9763@end deftypemethod
9764
9765@deftypemethod {YYParser} {boolean} getErrorVerbose ()
9766@deftypemethodx {YYParser} {void} setErrorVerbose (boolean @var{verbose})
9767Get or set the option to produce verbose error messages. These are only
9768available with @samp{%define parse.error verbose}, which also turns on
9769verbose error messages.
9770@end deftypemethod
9771
9772@deftypemethod {YYParser} {void} yyerror (String @var{msg})
9773@deftypemethodx {YYParser} {void} yyerror (Position @var{pos}, String @var{msg})
9774@deftypemethodx {YYParser} {void} yyerror (Location @var{loc}, String @var{msg})
9775Print an error message using the @code{yyerror} method of the scanner
9776instance in use. The @code{Location} and @code{Position} parameters are
9777available only if location tracking is active.
9778@end deftypemethod
9779
9780@deftypemethod {YYParser} {boolean} recovering ()
9781During the syntactic analysis, return @code{true} if recovering
9782from a syntax error.
9783@xref{Error Recovery}.
9784@end deftypemethod
9785
9786@deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
9787@deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
9788Get or set the stream used for tracing the parsing. It defaults to
9789@code{System.err}.
9790@end deftypemethod
9791
9792@deftypemethod {YYParser} {int} getDebugLevel ()
9793@deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
9794Get or set the tracing level. Currently its value is either 0, no trace,
9795or nonzero, full tracing.
9796@end deftypemethod
9797
9798@deftypecv {Constant} {YYParser} {String} {bisonVersion}
9799@deftypecvx {Constant} {YYParser} {String} {bisonSkeleton}
9800Identify the Bison version and skeleton used to generate this parser.
9801@end deftypecv
9802
9803
9804@node Java Scanner Interface
9805@subsection Java Scanner Interface
9806@c - %code lexer
9807@c - %lex-param
9808@c - Lexer interface
9809
9810There are two possible ways to interface a Bison-generated Java parser
9811with a scanner: the scanner may be defined by @code{%code lexer}, or
9812defined elsewhere. In either case, the scanner has to implement the
9813@code{Lexer} inner interface of the parser class. This interface also
9814contain constants for all user-defined token names and the predefined
9815@code{EOF} token.
9816
9817In the first case, the body of the scanner class is placed in
9818@code{%code lexer} blocks. If you want to pass parameters from the
9819parser constructor to the scanner constructor, specify them with
9820@code{%lex-param}; they are passed before @code{%parse-param}s to the
9821constructor.
9822
9823In the second case, the scanner has to implement the @code{Lexer} interface,
9824which is defined within the parser class (e.g., @code{YYParser.Lexer}).
9825The constructor of the parser object will then accept an object
9826implementing the interface; @code{%lex-param} is not used in this
9827case.
9828
9829In both cases, the scanner has to implement the following methods.
9830
9831@deftypemethod {Lexer} {void} yyerror (Location @var{loc}, String @var{msg})
9832This method is defined by the user to emit an error message. The first
9833parameter is omitted if location tracking is not active. Its type can be
9834changed using @samp{%define location_type "@var{class-name}".}
9835@end deftypemethod
9836
9837@deftypemethod {Lexer} {int} yylex ()
9838Return the next token. Its type is the return value, its semantic
9839value and location are saved and returned by the their methods in the
9840interface.
9841
9842Use @samp{%define lex_throws} to specify any uncaught exceptions.
9843Default is @code{java.io.IOException}.
9844@end deftypemethod
9845
9846@deftypemethod {Lexer} {Position} getStartPos ()
9847@deftypemethodx {Lexer} {Position} getEndPos ()
9848Return respectively the first position of the last token that
9849@code{yylex} returned, and the first position beyond it. These
9850methods are not needed unless location tracking is active.
9851
9852The return type can be changed using @samp{%define position_type
9853"@var{class-name}".}
9854@end deftypemethod
9855
9856@deftypemethod {Lexer} {Object} getLVal ()
9857Return the semantic value of the last token that yylex returned.
9858
9859The return type can be changed using @samp{%define stype
9860"@var{class-name}".}
9861@end deftypemethod
9862
9863
9864@node Java Action Features
9865@subsection Special Features for Use in Java Actions
9866
9867The following special constructs can be uses in Java actions.
9868Other analogous C action features are currently unavailable for Java.
9869
9870Use @samp{%define throws} to specify any uncaught exceptions from parser
9871actions, and initial actions specified by @code{%initial-action}.
9872
9873@defvar $@var{n}
9874The semantic value for the @var{n}th component of the current rule.
9875This may not be assigned to.
9876@xref{Java Semantic Values}.
9877@end defvar
9878
9879@defvar $<@var{typealt}>@var{n}
9880Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
9881@xref{Java Semantic Values}.
9882@end defvar
9883
9884@defvar $$
9885The semantic value for the grouping made by the current rule. As a
9886value, this is in the base type (@code{Object} or as specified by
9887@samp{%define stype}) as in not cast to the declared subtype because
9888casts are not allowed on the left-hand side of Java assignments.
9889Use an explicit Java cast if the correct subtype is needed.
9890@xref{Java Semantic Values}.
9891@end defvar
9892
9893@defvar $<@var{typealt}>$
9894Same as @code{$$} since Java always allow assigning to the base type.
9895Perhaps we should use this and @code{$<>$} for the value and @code{$$}
9896for setting the value but there is currently no easy way to distinguish
9897these constructs.
9898@xref{Java Semantic Values}.
9899@end defvar
9900
9901@defvar @@@var{n}
9902The location information of the @var{n}th component of the current rule.
9903This may not be assigned to.
9904@xref{Java Location Values}.
9905@end defvar
9906
9907@defvar @@$
9908The location information of the grouping made by the current rule.
9909@xref{Java Location Values}.
9910@end defvar
9911
9912@deffn {Statement} {return YYABORT;}
9913Return immediately from the parser, indicating failure.
9914@xref{Java Parser Interface}.
9915@end deffn
9916
9917@deffn {Statement} {return YYACCEPT;}
9918Return immediately from the parser, indicating success.
9919@xref{Java Parser Interface}.
9920@end deffn
9921
9922@deffn {Statement} {return YYERROR;}
9923Start error recovery without printing an error message.
9924@xref{Error Recovery}.
9925@end deffn
9926
9927@deffn {Statement} {return YYFAIL;}
9928Print an error message and start error recovery.
9929@xref{Error Recovery}.
9930@end deffn
9931
9932@deftypefn {Function} {boolean} recovering ()
9933Return whether error recovery is being done. In this state, the parser
9934reads token until it reaches a known state, and then restarts normal
9935operation.
9936@xref{Error Recovery}.
9937@end deftypefn
9938
9939@deftypefn {Function} {void} yyerror (String @var{msg})
9940@deftypefnx {Function} {void} yyerror (Position @var{loc}, String @var{msg})
9941@deftypefnx {Function} {void} yyerror (Location @var{loc}, String @var{msg})
9942Print an error message using the @code{yyerror} method of the scanner
9943instance in use. The @code{Location} and @code{Position} parameters are
9944available only if location tracking is active.
9945@end deftypefn
9946
9947
9948@node Java Differences
9949@subsection Differences between C/C++ and Java Grammars
9950
9951The different structure of the Java language forces several differences
9952between C/C++ grammars, and grammars designed for Java parsers. This
9953section summarizes these differences.
9954
9955@itemize
9956@item
9957Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
9958@code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
9959macros. Instead, they should be preceded by @code{return} when they
9960appear in an action. The actual definition of these symbols is
9961opaque to the Bison grammar, and it might change in the future. The
9962only meaningful operation that you can do, is to return them.
9963See @pxref{Java Action Features}.
9964
9965Note that of these three symbols, only @code{YYACCEPT} and
9966@code{YYABORT} will cause a return from the @code{yyparse}
9967method@footnote{Java parsers include the actions in a separate
9968method than @code{yyparse} in order to have an intuitive syntax that
9969corresponds to these C macros.}.
9970
9971@item
9972Java lacks unions, so @code{%union} has no effect. Instead, semantic
9973values have a common base type: @code{Object} or as specified by
9974@samp{%define stype}. Angle brackets on @code{%token}, @code{type},
9975@code{$@var{n}} and @code{$$} specify subtypes rather than fields of
9976an union. The type of @code{$$}, even with angle brackets, is the base
9977type since Java casts are not allow on the left-hand side of assignments.
9978Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
9979left-hand side of assignments. See @pxref{Java Semantic Values} and
9980@pxref{Java Action Features}.
9981
9982@item
9983The prologue declarations have a different meaning than in C/C++ code.
9984@table @asis
9985@item @code{%code imports}
9986blocks are placed at the beginning of the Java source code. They may
9987include copyright notices. For a @code{package} declarations, it is
9988suggested to use @samp{%define package} instead.
9989
9990@item unqualified @code{%code}
9991blocks are placed inside the parser class.
9992
9993@item @code{%code lexer}
9994blocks, if specified, should include the implementation of the
9995scanner. If there is no such block, the scanner can be any class
9996that implements the appropriate interface (see @pxref{Java Scanner
9997Interface}).
9998@end table
9999
10000Other @code{%code} blocks are not supported in Java parsers.
10001In particular, @code{%@{ @dots{} %@}} blocks should not be used
10002and may give an error in future versions of Bison.
10003
10004The epilogue has the same meaning as in C/C++ code and it can
10005be used to define other classes used by the parser @emph{outside}
10006the parser class.
10007@end itemize
10008
10009
10010@node Java Declarations Summary
10011@subsection Java Declarations Summary
10012
10013This summary only include declarations specific to Java or have special
10014meaning when used in a Java parser.
10015
10016@deffn {Directive} {%language "Java"}
10017Generate a Java class for the parser.
10018@end deffn
10019
10020@deffn {Directive} %lex-param @{@var{type} @var{name}@}
10021A parameter for the lexer class defined by @code{%code lexer}
10022@emph{only}, added as parameters to the lexer constructor and the parser
10023constructor that @emph{creates} a lexer. Default is none.
10024@xref{Java Scanner Interface}.
10025@end deffn
10026
10027@deffn {Directive} %name-prefix "@var{prefix}"
10028The prefix of the parser class name @code{@var{prefix}Parser} if
10029@samp{%define parser_class_name} is not used. Default is @code{YY}.
10030@xref{Java Bison Interface}.
10031@end deffn
10032
10033@deffn {Directive} %parse-param @{@var{type} @var{name}@}
10034A parameter for the parser class added as parameters to constructor(s)
10035and as fields initialized by the constructor(s). Default is none.
10036@xref{Java Parser Interface}.
10037@end deffn
10038
10039@deffn {Directive} %token <@var{type}> @var{token} @dots{}
10040Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
10041@xref{Java Semantic Values}.
10042@end deffn
10043
10044@deffn {Directive} %type <@var{type}> @var{nonterminal} @dots{}
10045Declare the type of nonterminals. Note that the angle brackets enclose
10046a Java @emph{type}.
10047@xref{Java Semantic Values}.
10048@end deffn
10049
10050@deffn {Directive} %code @{ @var{code} @dots{} @}
10051Code appended to the inside of the parser class.
10052@xref{Java Differences}.
10053@end deffn
10054
10055@deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
10056Code inserted just after the @code{package} declaration.
10057@xref{Java Differences}.
10058@end deffn
10059
10060@deffn {Directive} {%code init} @{ @var{code} @dots{} @}
10061Code inserted at the beginning of the parser constructor body.
10062@xref{Java Parser Interface}.
10063@end deffn
10064
10065@deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
10066Code added to the body of a inner lexer class within the parser class.
10067@xref{Java Scanner Interface}.
10068@end deffn
10069
10070@deffn {Directive} %% @var{code} @dots{}
10071Code (after the second @code{%%}) appended to the end of the file,
10072@emph{outside} the parser class.
10073@xref{Java Differences}.
10074@end deffn
10075
10076@deffn {Directive} %@{ @var{code} @dots{} %@}
10077Not supported. Use @code{%code imports} instead.
10078@xref{Java Differences}.
10079@end deffn
10080
10081@deffn {Directive} {%define abstract}
10082Whether the parser class is declared @code{abstract}. Default is false.
10083@xref{Java Bison Interface}.
10084@end deffn
10085
10086@deffn {Directive} {%define annotations} "@var{annotations}"
10087The Java annotations for the parser class. Default is none.
10088@xref{Java Bison Interface}.
10089@end deffn
10090
10091@deffn {Directive} {%define extends} "@var{superclass}"
10092The superclass of the parser class. Default is none.
10093@xref{Java Bison Interface}.
10094@end deffn
10095
10096@deffn {Directive} {%define final}
10097Whether the parser class is declared @code{final}. Default is false.
10098@xref{Java Bison Interface}.
10099@end deffn
10100
10101@deffn {Directive} {%define implements} "@var{interfaces}"
10102The implemented interfaces of the parser class, a comma-separated list.
10103Default is none.
10104@xref{Java Bison Interface}.
10105@end deffn
10106
10107@deffn {Directive} {%define init_throws} "@var{exceptions}"
10108The exceptions thrown by @code{%code init} from the parser class
10109constructor. Default is none.
10110@xref{Java Parser Interface}.
10111@end deffn
10112
10113@deffn {Directive} {%define lex_throws} "@var{exceptions}"
10114The exceptions thrown by the @code{yylex} method of the lexer, a
10115comma-separated list. Default is @code{java.io.IOException}.
10116@xref{Java Scanner Interface}.
10117@end deffn
10118
10119@deffn {Directive} {%define location_type} "@var{class}"
10120The name of the class used for locations (a range between two
10121positions). This class is generated as an inner class of the parser
10122class by @command{bison}. Default is @code{Location}.
10123@xref{Java Location Values}.
10124@end deffn
10125
10126@deffn {Directive} {%define package} "@var{package}"
10127The package to put the parser class in. Default is none.
10128@xref{Java Bison Interface}.
10129@end deffn
10130
10131@deffn {Directive} {%define parser_class_name} "@var{name}"
10132The name of the parser class. Default is @code{YYParser} or
10133@code{@var{name-prefix}Parser}.
10134@xref{Java Bison Interface}.
10135@end deffn
10136
10137@deffn {Directive} {%define position_type} "@var{class}"
10138The name of the class used for positions. This class must be supplied by
10139the user. Default is @code{Position}.
10140@xref{Java Location Values}.
10141@end deffn
10142
10143@deffn {Directive} {%define public}
10144Whether the parser class is declared @code{public}. Default is false.
10145@xref{Java Bison Interface}.
10146@end deffn
10147
10148@deffn {Directive} {%define stype} "@var{class}"
10149The base type of semantic values. Default is @code{Object}.
10150@xref{Java Semantic Values}.
10151@end deffn
10152
10153@deffn {Directive} {%define strictfp}
10154Whether the parser class is declared @code{strictfp}. Default is false.
10155@xref{Java Bison Interface}.
10156@end deffn
10157
10158@deffn {Directive} {%define throws} "@var{exceptions}"
10159The exceptions thrown by user-supplied parser actions and
10160@code{%initial-action}, a comma-separated list. Default is none.
10161@xref{Java Parser Interface}.
10162@end deffn
10163
10164
10165@c ================================================= FAQ
10166
10167@node FAQ
10168@chapter Frequently Asked Questions
10169@cindex frequently asked questions
10170@cindex questions
10171
10172Several questions about Bison come up occasionally. Here some of them
10173are addressed.
10174
10175@menu
10176* Memory Exhausted:: Breaking the Stack Limits
10177* How Can I Reset the Parser:: @code{yyparse} Keeps some State
10178* Strings are Destroyed:: @code{yylval} Loses Track of Strings
10179* Implementing Gotos/Loops:: Control Flow in the Calculator
10180* Multiple start-symbols:: Factoring closely related grammars
10181* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
10182* I can't build Bison:: Troubleshooting
10183* Where can I find help?:: Troubleshouting
10184* Bug Reports:: Troublereporting
10185* More Languages:: Parsers in C++, Java, and so on
10186* Beta Testing:: Experimenting development versions
10187* Mailing Lists:: Meeting other Bison users
10188@end menu
10189
10190@node Memory Exhausted
10191@section Memory Exhausted
10192
10193@display
10194My parser returns with error with a @samp{memory exhausted}
10195message. What can I do?
10196@end display
10197
10198This question is already addressed elsewhere, @xref{Recursion,
10199,Recursive Rules}.
10200
10201@node How Can I Reset the Parser
10202@section How Can I Reset the Parser
10203
10204The following phenomenon has several symptoms, resulting in the
10205following typical questions:
10206
10207@display
10208I invoke @code{yyparse} several times, and on correct input it works
10209properly; but when a parse error is found, all the other calls fail
10210too. How can I reset the error flag of @code{yyparse}?
10211@end display
10212
10213@noindent
10214or
10215
10216@display
10217My parser includes support for an @samp{#include}-like feature, in
10218which case I run @code{yyparse} from @code{yyparse}. This fails
10219although I did specify @samp{%define api.pure}.
10220@end display
10221
10222These problems typically come not from Bison itself, but from
10223Lex-generated scanners. Because these scanners use large buffers for
10224speed, they might not notice a change of input file. As a
10225demonstration, consider the following source file,
10226@file{first-line.l}:
10227
10228@verbatim
10229%{
10230#include <stdio.h>
10231#include <stdlib.h>
10232%}
10233%%
10234.*\n ECHO; return 1;
10235%%
10236int
10237yyparse (char const *file)
10238{
10239 yyin = fopen (file, "r");
10240 if (!yyin)
10241 exit (2);
10242 /* One token only. */
10243 yylex ();
10244 if (fclose (yyin) != 0)
10245 exit (3);
10246 return 0;
10247}
10248
10249int
10250main (void)
10251{
10252 yyparse ("input");
10253 yyparse ("input");
10254 return 0;
10255}
10256@end verbatim
10257
10258@noindent
10259If the file @file{input} contains
10260
10261@verbatim
10262input:1: Hello,
10263input:2: World!
10264@end verbatim
10265
10266@noindent
10267then instead of getting the first line twice, you get:
10268
10269@example
10270$ @kbd{flex -ofirst-line.c first-line.l}
10271$ @kbd{gcc -ofirst-line first-line.c -ll}
10272$ @kbd{./first-line}
10273input:1: Hello,
10274input:2: World!
10275@end example
10276
10277Therefore, whenever you change @code{yyin}, you must tell the
10278Lex-generated scanner to discard its current buffer and switch to the
10279new one. This depends upon your implementation of Lex; see its
10280documentation for more. For Flex, it suffices to call
10281@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
10282Flex-generated scanner needs to read from several input streams to
10283handle features like include files, you might consider using Flex
10284functions like @samp{yy_switch_to_buffer} that manipulate multiple
10285input buffers.
10286
10287If your Flex-generated scanner uses start conditions (@pxref{Start
10288conditions, , Start conditions, flex, The Flex Manual}), you might
10289also want to reset the scanner's state, i.e., go back to the initial
10290start condition, through a call to @samp{BEGIN (0)}.
10291
10292@node Strings are Destroyed
10293@section Strings are Destroyed
10294
10295@display
10296My parser seems to destroy old strings, or maybe it loses track of
10297them. Instead of reporting @samp{"foo", "bar"}, it reports
10298@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
10299@end display
10300
10301This error is probably the single most frequent ``bug report'' sent to
10302Bison lists, but is only concerned with a misunderstanding of the role
10303of the scanner. Consider the following Lex code:
10304
10305@verbatim
10306%{
10307#include <stdio.h>
10308char *yylval = NULL;
10309%}
10310%%
10311.* yylval = yytext; return 1;
10312\n /* IGNORE */
10313%%
10314int
10315main ()
10316{
10317 /* Similar to using $1, $2 in a Bison action. */
10318 char *fst = (yylex (), yylval);
10319 char *snd = (yylex (), yylval);
10320 printf ("\"%s\", \"%s\"\n", fst, snd);
10321 return 0;
10322}
10323@end verbatim
10324
10325If you compile and run this code, you get:
10326
10327@example
10328$ @kbd{flex -osplit-lines.c split-lines.l}
10329$ @kbd{gcc -osplit-lines split-lines.c -ll}
10330$ @kbd{printf 'one\ntwo\n' | ./split-lines}
10331"one
10332two", "two"
10333@end example
10334
10335@noindent
10336this is because @code{yytext} is a buffer provided for @emph{reading}
10337in the action, but if you want to keep it, you have to duplicate it
10338(e.g., using @code{strdup}). Note that the output may depend on how
10339your implementation of Lex handles @code{yytext}. For instance, when
10340given the Lex compatibility option @option{-l} (which triggers the
10341option @samp{%array}) Flex generates a different behavior:
10342
10343@example
10344$ @kbd{flex -l -osplit-lines.c split-lines.l}
10345$ @kbd{gcc -osplit-lines split-lines.c -ll}
10346$ @kbd{printf 'one\ntwo\n' | ./split-lines}
10347"two", "two"
10348@end example
10349
10350
10351@node Implementing Gotos/Loops
10352@section Implementing Gotos/Loops
10353
10354@display
10355My simple calculator supports variables, assignments, and functions,
10356but how can I implement gotos, or loops?
10357@end display
10358
10359Although very pedagogical, the examples included in the document blur
10360the distinction to make between the parser---whose job is to recover
10361the structure of a text and to transmit it to subsequent modules of
10362the program---and the processing (such as the execution) of this
10363structure. This works well with so called straight line programs,
10364i.e., precisely those that have a straightforward execution model:
10365execute simple instructions one after the others.
10366
10367@cindex abstract syntax tree
10368@cindex @acronym{AST}
10369If you want a richer model, you will probably need to use the parser
10370to construct a tree that does represent the structure it has
10371recovered; this tree is usually called the @dfn{abstract syntax tree},
10372or @dfn{@acronym{AST}} for short. Then, walking through this tree,
10373traversing it in various ways, will enable treatments such as its
10374execution or its translation, which will result in an interpreter or a
10375compiler.
10376
10377This topic is way beyond the scope of this manual, and the reader is
10378invited to consult the dedicated literature.
10379
10380
10381@node Multiple start-symbols
10382@section Multiple start-symbols
10383
10384@display
10385I have several closely related grammars, and I would like to share their
10386implementations. In fact, I could use a single grammar but with
10387multiple entry points.
10388@end display
10389
10390Bison does not support multiple start-symbols, but there is a very
10391simple means to simulate them. If @code{foo} and @code{bar} are the two
10392pseudo start-symbols, then introduce two new tokens, say
10393@code{START_FOO} and @code{START_BAR}, and use them as switches from the
10394real start-symbol:
10395
10396@example
10397%token START_FOO START_BAR;
10398%start start;
10399start: START_FOO foo
10400 | START_BAR bar;
10401@end example
10402
10403These tokens prevents the introduction of new conflicts. As far as the
10404parser goes, that is all that is needed.
10405
10406Now the difficult part is ensuring that the scanner will send these
10407tokens first. If your scanner is hand-written, that should be
10408straightforward. If your scanner is generated by Lex, them there is
10409simple means to do it: recall that anything between @samp{%@{ ... %@}}
10410after the first @code{%%} is copied verbatim in the top of the generated
10411@code{yylex} function. Make sure a variable @code{start_token} is
10412available in the scanner (e.g., a global variable or using
10413@code{%lex-param} etc.), and use the following:
10414
10415@example
10416 /* @r{Prologue.} */
10417%%
10418%@{
10419 if (start_token)
10420 @{
10421 int t = start_token;
10422 start_token = 0;
10423 return t;
10424 @}
10425%@}
10426 /* @r{The rules.} */
10427@end example
10428
10429
10430@node Secure? Conform?
10431@section Secure? Conform?
10432
10433@display
10434Is Bison secure? Does it conform to POSIX?
10435@end display
10436
10437If you're looking for a guarantee or certification, we don't provide it.
10438However, Bison is intended to be a reliable program that conforms to the
10439@acronym{POSIX} specification for Yacc. If you run into problems,
10440please send us a bug report.
10441
10442@node I can't build Bison
10443@section I can't build Bison
10444
10445@display
10446I can't build Bison because @command{make} complains that
10447@code{msgfmt} is not found.
10448What should I do?
10449@end display
10450
10451Like most GNU packages with internationalization support, that feature
10452is turned on by default. If you have problems building in the @file{po}
10453subdirectory, it indicates that your system's internationalization
10454support is lacking. You can re-configure Bison with
10455@option{--disable-nls} to turn off this support, or you can install GNU
10456gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
10457Bison. See the file @file{ABOUT-NLS} for more information.
10458
10459
10460@node Where can I find help?
10461@section Where can I find help?
10462
10463@display
10464I'm having trouble using Bison. Where can I find help?
10465@end display
10466
10467First, read this fine manual. Beyond that, you can send mail to
10468@email{help-bison@@gnu.org}. This mailing list is intended to be
10469populated with people who are willing to answer questions about using
10470and installing Bison. Please keep in mind that (most of) the people on
10471the list have aspects of their lives which are not related to Bison (!),
10472so you may not receive an answer to your question right away. This can
10473be frustrating, but please try not to honk them off; remember that any
10474help they provide is purely voluntary and out of the kindness of their
10475hearts.
10476
10477@node Bug Reports
10478@section Bug Reports
10479
10480@display
10481I found a bug. What should I include in the bug report?
10482@end display
10483
10484Before you send a bug report, make sure you are using the latest
10485version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
10486mirrors. Be sure to include the version number in your bug report. If
10487the bug is present in the latest version but not in a previous version,
10488try to determine the most recent version which did not contain the bug.
10489
10490If the bug is parser-related, you should include the smallest grammar
10491you can which demonstrates the bug. The grammar file should also be
10492complete (i.e., I should be able to run it through Bison without having
10493to edit or add anything). The smaller and simpler the grammar, the
10494easier it will be to fix the bug.
10495
10496Include information about your compilation environment, including your
10497operating system's name and version and your compiler's name and
10498version. If you have trouble compiling, you should also include a
10499transcript of the build session, starting with the invocation of
10500`configure'. Depending on the nature of the bug, you may be asked to
10501send additional files as well (such as `config.h' or `config.cache').
10502
10503Patches are most welcome, but not required. That is, do not hesitate to
10504send a bug report just because you can not provide a fix.
10505
10506Send bug reports to @email{bug-bison@@gnu.org}.
10507
10508@node More Languages
10509@section More Languages
10510
10511@display
10512Will Bison ever have C++ and Java support? How about @var{insert your
10513favorite language here}?
10514@end display
10515
10516C++ and Java support is there now, and is documented. We'd love to add other
10517languages; contributions are welcome.
10518
10519@node Beta Testing
10520@section Beta Testing
10521
10522@display
10523What is involved in being a beta tester?
10524@end display
10525
10526It's not terribly involved. Basically, you would download a test
10527release, compile it, and use it to build and run a parser or two. After
10528that, you would submit either a bug report or a message saying that
10529everything is okay. It is important to report successes as well as
10530failures because test releases eventually become mainstream releases,
10531but only if they are adequately tested. If no one tests, development is
10532essentially halted.
10533
10534Beta testers are particularly needed for operating systems to which the
10535developers do not have easy access. They currently have easy access to
10536recent GNU/Linux and Solaris versions. Reports about other operating
10537systems are especially welcome.
10538
10539@node Mailing Lists
10540@section Mailing Lists
10541
10542@display
10543How do I join the help-bison and bug-bison mailing lists?
10544@end display
10545
10546See @url{http://lists.gnu.org/}.
10547
10548@c ================================================= Table of Symbols
10549
10550@node Table of Symbols
10551@appendix Bison Symbols
10552@cindex Bison symbols, table of
10553@cindex symbols in Bison, table of
10554
10555@deffn {Variable} @@$
10556In an action, the location of the left-hand side of the rule.
10557@xref{Locations, , Locations Overview}.
10558@end deffn
10559
10560@deffn {Variable} @@@var{n}
10561In an action, the location of the @var{n}-th symbol of the right-hand
10562side of the rule. @xref{Locations, , Locations Overview}.
10563@end deffn
10564
10565@deffn {Variable} @@@var{name}
10566In an action, the location of a symbol addressed by name.
10567@xref{Locations, , Locations Overview}.
10568@end deffn
10569
10570@deffn {Variable} @@[@var{name}]
10571In an action, the location of a symbol addressed by name.
10572@xref{Locations, , Locations Overview}.
10573@end deffn
10574
10575@deffn {Variable} $$
10576In an action, the semantic value of the left-hand side of the rule.
10577@xref{Actions}.
10578@end deffn
10579
10580@deffn {Variable} $@var{n}
10581In an action, the semantic value of the @var{n}-th symbol of the
10582right-hand side of the rule. @xref{Actions}.
10583@end deffn
10584
10585@deffn {Variable} $@var{name}
10586In an action, the semantic value of a symbol addressed by name.
10587@xref{Actions}.
10588@end deffn
10589
10590@deffn {Variable} $[@var{name}]
10591In an action, the semantic value of a symbol addressed by name.
10592@xref{Actions}.
10593@end deffn
10594
10595@deffn {Delimiter} %%
10596Delimiter used to separate the grammar rule section from the
10597Bison declarations section or the epilogue.
10598@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
10599@end deffn
10600
10601@c Don't insert spaces, or check the DVI output.
10602@deffn {Delimiter} %@{@var{code}%@}
10603All code listed between @samp{%@{} and @samp{%@}} is copied directly to
10604the output file uninterpreted. Such code forms the prologue of the input
10605file. @xref{Grammar Outline, ,Outline of a Bison
10606Grammar}.
10607@end deffn
10608
10609@deffn {Construct} /*@dots{}*/
10610Comment delimiters, as in C.
10611@end deffn
10612
10613@deffn {Delimiter} :
10614Separates a rule's result from its components. @xref{Rules, ,Syntax of
10615Grammar Rules}.
10616@end deffn
10617
10618@deffn {Delimiter} ;
10619Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
10620@end deffn
10621
10622@deffn {Delimiter} |
10623Separates alternate rules for the same result nonterminal.
10624@xref{Rules, ,Syntax of Grammar Rules}.
10625@end deffn
10626
10627@deffn {Directive} <*>
10628Used to define a default tagged @code{%destructor} or default tagged
10629@code{%printer}.
10630
10631This feature is experimental.
10632More user feedback will help to determine whether it should become a permanent
10633feature.
10634
10635@xref{Destructor Decl, , Freeing Discarded Symbols}.
10636@end deffn
10637
10638@deffn {Directive} <>
10639Used to define a default tagless @code{%destructor} or default tagless
10640@code{%printer}.
10641
10642This feature is experimental.
10643More user feedback will help to determine whether it should become a permanent
10644feature.
10645
10646@xref{Destructor Decl, , Freeing Discarded Symbols}.
10647@end deffn
10648
10649@deffn {Symbol} $accept
10650The predefined nonterminal whose only rule is @samp{$accept: @var{start}
10651$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
10652Start-Symbol}. It cannot be used in the grammar.
10653@end deffn
10654
10655@deffn {Directive} %code @{@var{code}@}
10656@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
10657Insert @var{code} verbatim into output parser source.
10658@xref{Decl Summary,,%code}.
10659@end deffn
10660
10661@deffn {Directive} %debug
10662Equip the parser for debugging. @xref{Decl Summary}.
10663@end deffn
10664
10665@ifset defaultprec
10666@deffn {Directive} %default-prec
10667Assign a precedence to rules that lack an explicit @samp{%prec}
10668modifier. @xref{Contextual Precedence, ,Context-Dependent
10669Precedence}.
10670@end deffn
10671@end ifset
10672
10673@deffn {Directive} %define @var{define-variable}
10674@deffnx {Directive} %define @var{define-variable} @var{value}
10675@deffnx {Directive} %define @var{define-variable} "@var{value}"
10676Define a variable to adjust Bison's behavior.
10677@xref{Decl Summary,,%define}.
10678@end deffn
10679
10680@deffn {Directive} %defines
10681Bison declaration to create a header file meant for the scanner.
10682@xref{Decl Summary}.
10683@end deffn
10684
10685@deffn {Directive} %defines @var{defines-file}
10686Same as above, but save in the file @var{defines-file}.
10687@xref{Decl Summary}.
10688@end deffn
10689
10690@deffn {Directive} %destructor
10691Specify how the parser should reclaim the memory associated to
10692discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
10693@end deffn
10694
10695@deffn {Directive} %dprec
10696Bison declaration to assign a precedence to a rule that is used at parse
10697time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
10698@acronym{GLR} Parsers}.
10699@end deffn
10700
10701@deffn {Symbol} $end
10702The predefined token marking the end of the token stream. It cannot be
10703used in the grammar.
10704@end deffn
10705
10706@deffn {Symbol} error
10707A token name reserved for error recovery. This token may be used in
10708grammar rules so as to allow the Bison parser to recognize an error in
10709the grammar without halting the process. In effect, a sentence
10710containing an error may be recognized as valid. On a syntax error, the
10711token @code{error} becomes the current lookahead token. Actions
10712corresponding to @code{error} are then executed, and the lookahead
10713token is reset to the token that originally caused the violation.
10714@xref{Error Recovery}.
10715@end deffn
10716
10717@deffn {Directive} %error-verbose
10718An obsolete directive standing for @samp{%define parse.error verbose}.
10719@end deffn
10720
10721@deffn {Directive} %file-prefix "@var{prefix}"
10722Bison declaration to set the prefix of the output files. @xref{Decl
10723Summary}.
10724@end deffn
10725
10726@deffn {Directive} %glr-parser
10727Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
10728Parsers, ,Writing @acronym{GLR} Parsers}.
10729@end deffn
10730
10731@deffn {Directive} %initial-action
10732Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
10733@end deffn
10734
10735@deffn {Directive} %language
10736Specify the programming language for the generated parser.
10737@xref{Decl Summary}.
10738@end deffn
10739
10740@deffn {Directive} %left
10741Bison declaration to assign precedence and left associativity to token(s).
10742@xref{Precedence Decl, ,Operator Precedence}.
10743@end deffn
10744
10745@deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
10746Bison declaration to specifying additional arguments that
10747@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
10748for Pure Parsers}.
10749@end deffn
10750
10751@deffn {Directive} %merge
10752Bison declaration to assign a merging function to a rule. If there is a
10753reduce/reduce conflict with a rule having the same merging function, the
10754function is applied to the two semantic values to get a single result.
10755@xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
10756@end deffn
10757
10758@deffn {Directive} %name-prefix "@var{prefix}"
10759Bison declaration to rename the external symbols. @xref{Decl Summary}.
10760@end deffn
10761
10762@ifset defaultprec
10763@deffn {Directive} %no-default-prec
10764Do not assign a precedence to rules that lack an explicit @samp{%prec}
10765modifier. @xref{Contextual Precedence, ,Context-Dependent
10766Precedence}.
10767@end deffn
10768@end ifset
10769
10770@deffn {Directive} %no-lines
10771Bison declaration to avoid generating @code{#line} directives in the
10772parser file. @xref{Decl Summary}.
10773@end deffn
10774
10775@deffn {Directive} %nonassoc
10776Bison declaration to assign precedence and nonassociativity to token(s).
10777@xref{Precedence Decl, ,Operator Precedence}.
10778@end deffn
10779
10780@deffn {Directive} %output "@var{file}"
10781Bison declaration to set the name of the parser file. @xref{Decl
10782Summary}.
10783@end deffn
10784
10785@deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
10786Bison declaration to specify additional arguments that both
10787@code{yylex} and @code{yyparse} should accept. @xref{Parser Function,, The
10788Parser Function @code{yyparse}}.
10789@end deffn
10790
10791@deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
10792Bison declaration to specify additional arguments that @code{yyparse}
10793should accept. @xref{Parser Function,, The Parser Function @code{yyparse}}.
10794@end deffn
10795
10796@deffn {Directive} %prec
10797Bison declaration to assign a precedence to a specific rule.
10798@xref{Contextual Precedence, ,Context-Dependent Precedence}.
10799@end deffn
10800
10801@deffn {Directive} %precedence
10802Bison declaration to assign precedence to token(s), but no associativity
10803@xref{Precedence Decl, ,Operator Precedence}.
10804@end deffn
10805
10806@deffn {Directive} %pure-parser
10807Deprecated version of @samp{%define api.pure} (@pxref{Decl Summary, ,%define}),
10808for which Bison is more careful to warn about unreasonable usage.
10809@end deffn
10810
10811@deffn {Directive} %require "@var{version}"
10812Require version @var{version} or higher of Bison. @xref{Require Decl, ,
10813Require a Version of Bison}.
10814@end deffn
10815
10816@deffn {Directive} %right
10817Bison declaration to assign precedence and right associativity to token(s).
10818@xref{Precedence Decl, ,Operator Precedence}.
10819@end deffn
10820
10821@deffn {Directive} %skeleton
10822Specify the skeleton to use; usually for development.
10823@xref{Decl Summary}.
10824@end deffn
10825
10826@deffn {Directive} %start
10827Bison declaration to specify the start symbol. @xref{Start Decl, ,The
10828Start-Symbol}.
10829@end deffn
10830
10831@deffn {Directive} %token
10832Bison declaration to declare token(s) without specifying precedence.
10833@xref{Token Decl, ,Token Type Names}.
10834@end deffn
10835
10836@deffn {Directive} %token-table
10837Bison declaration to include a token name table in the parser file.
10838@xref{Decl Summary}.
10839@end deffn
10840
10841@deffn {Directive} %type
10842Bison declaration to declare nonterminals. @xref{Type Decl,
10843,Nonterminal Symbols}.
10844@end deffn
10845
10846@deffn {Symbol} $undefined
10847The predefined token onto which all undefined values returned by
10848@code{yylex} are mapped. It cannot be used in the grammar, rather, use
10849@code{error}.
10850@end deffn
10851
10852@deffn {Directive} %union
10853Bison declaration to specify several possible data types for semantic
10854values. @xref{Union Decl, ,The Collection of Value Types}.
10855@end deffn
10856
10857@deffn {Macro} YYABORT
10858Macro to pretend that an unrecoverable syntax error has occurred, by
10859making @code{yyparse} return 1 immediately. The error reporting
10860function @code{yyerror} is not called. @xref{Parser Function, ,The
10861Parser Function @code{yyparse}}.
10862
10863For Java parsers, this functionality is invoked using @code{return YYABORT;}
10864instead.
10865@end deffn
10866
10867@deffn {Macro} YYACCEPT
10868Macro to pretend that a complete utterance of the language has been
10869read, by making @code{yyparse} return 0 immediately.
10870@xref{Parser Function, ,The Parser Function @code{yyparse}}.
10871
10872For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
10873instead.
10874@end deffn
10875
10876@deffn {Macro} YYBACKUP
10877Macro to discard a value from the parser stack and fake a lookahead
10878token. @xref{Action Features, ,Special Features for Use in Actions}.
10879@end deffn
10880
10881@deffn {Variable} yychar
10882External integer variable that contains the integer value of the
10883lookahead token. (In a pure parser, it is a local variable within
10884@code{yyparse}.) Error-recovery rule actions may examine this variable.
10885@xref{Action Features, ,Special Features for Use in Actions}.
10886@end deffn
10887
10888@deffn {Variable} yyclearin
10889Macro used in error-recovery rule actions. It clears the previous
10890lookahead token. @xref{Error Recovery}.
10891@end deffn
10892
10893@deffn {Macro} YYDEBUG
10894Macro to define to equip the parser with tracing code. @xref{Tracing,
10895,Tracing Your Parser}.
10896@end deffn
10897
10898@deffn {Variable} yydebug
10899External integer variable set to zero by default. If @code{yydebug}
10900is given a nonzero value, the parser will output information on input
10901symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
10902@end deffn
10903
10904@deffn {Macro} yyerrok
10905Macro to cause parser to recover immediately to its normal mode
10906after a syntax error. @xref{Error Recovery}.
10907@end deffn
10908
10909@deffn {Macro} YYERROR
10910Macro to pretend that a syntax error has just been detected: call
10911@code{yyerror} and then perform normal error recovery if possible
10912(@pxref{Error Recovery}), or (if recovery is impossible) make
10913@code{yyparse} return 1. @xref{Error Recovery}.
10914
10915For Java parsers, this functionality is invoked using @code{return YYERROR;}
10916instead.
10917@end deffn
10918
10919@deffn {Function} yyerror
10920User-supplied function to be called by @code{yyparse} on error.
10921@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
10922@end deffn
10923
10924@deffn {Macro} YYERROR_VERBOSE
10925An obsolete macro used in the @file{yacc.c} skeleton, that you define
10926with @code{#define} in the prologue to request verbose, specific error
10927message strings when @code{yyerror} is called. It doesn't matter what
10928definition you use for @code{YYERROR_VERBOSE}, just whether you define
10929it. Using @samp{%define parse.error verbose} is preferred
10930(@pxref{Error Reporting, ,The Error Reporting Function @code{yyerror}}).
10931@end deffn
10932
10933@deffn {Macro} YYINITDEPTH
10934Macro for specifying the initial size of the parser stack.
10935@xref{Memory Management}.
10936@end deffn
10937
10938@deffn {Function} yylex
10939User-supplied lexical analyzer function, called with no arguments to get
10940the next token. @xref{Lexical, ,The Lexical Analyzer Function
10941@code{yylex}}.
10942@end deffn
10943
10944@deffn {Macro} YYLEX_PARAM
10945An obsolete macro for specifying an extra argument (or list of extra
10946arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
10947macro is deprecated, and is supported only for Yacc like parsers.
10948@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
10949@end deffn
10950
10951@deffn {Variable} yylloc
10952External variable in which @code{yylex} should place the line and column
10953numbers associated with a token. (In a pure parser, it is a local
10954variable within @code{yyparse}, and its address is passed to
10955@code{yylex}.)
10956You can ignore this variable if you don't use the @samp{@@} feature in the
10957grammar actions.
10958@xref{Token Locations, ,Textual Locations of Tokens}.
10959In semantic actions, it stores the location of the lookahead token.
10960@xref{Actions and Locations, ,Actions and Locations}.
10961@end deffn
10962
10963@deffn {Type} YYLTYPE
10964Data type of @code{yylloc}; by default, a structure with four
10965members. @xref{Location Type, , Data Types of Locations}.
10966@end deffn
10967
10968@deffn {Variable} yylval
10969External variable in which @code{yylex} should place the semantic
10970value associated with a token. (In a pure parser, it is a local
10971variable within @code{yyparse}, and its address is passed to
10972@code{yylex}.)
10973@xref{Token Values, ,Semantic Values of Tokens}.
10974In semantic actions, it stores the semantic value of the lookahead token.
10975@xref{Actions, ,Actions}.
10976@end deffn
10977
10978@deffn {Macro} YYMAXDEPTH
10979Macro for specifying the maximum size of the parser stack. @xref{Memory
10980Management}.
10981@end deffn
10982
10983@deffn {Variable} yynerrs
10984Global variable which Bison increments each time it reports a syntax error.
10985(In a pure parser, it is a local variable within @code{yyparse}. In a
10986pure push parser, it is a member of yypstate.)
10987@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
10988@end deffn
10989
10990@deffn {Function} yyparse
10991The parser function produced by Bison; call this function to start
10992parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
10993@end deffn
10994
10995@deffn {Function} yypstate_delete
10996The function to delete a parser instance, produced by Bison in push mode;
10997call this function to delete the memory associated with a parser.
10998@xref{Parser Delete Function, ,The Parser Delete Function
10999@code{yypstate_delete}}.
11000(The current push parsing interface is experimental and may evolve.
11001More user feedback will help to stabilize it.)
11002@end deffn
11003
11004@deffn {Function} yypstate_new
11005The function to create a parser instance, produced by Bison in push mode;
11006call this function to create a new parser.
11007@xref{Parser Create Function, ,The Parser Create Function
11008@code{yypstate_new}}.
11009(The current push parsing interface is experimental and may evolve.
11010More user feedback will help to stabilize it.)
11011@end deffn
11012
11013@deffn {Function} yypull_parse
11014The parser function produced by Bison in push mode; call this function to
11015parse the rest of the input stream.
11016@xref{Pull Parser Function, ,The Pull Parser Function
11017@code{yypull_parse}}.
11018(The current push parsing interface is experimental and may evolve.
11019More user feedback will help to stabilize it.)
11020@end deffn
11021
11022@deffn {Function} yypush_parse
11023The parser function produced by Bison in push mode; call this function to
11024parse a single token. @xref{Push Parser Function, ,The Push Parser Function
11025@code{yypush_parse}}.
11026(The current push parsing interface is experimental and may evolve.
11027More user feedback will help to stabilize it.)
11028@end deffn
11029
11030@deffn {Macro} YYPARSE_PARAM
11031An obsolete macro for specifying the name of a parameter that
11032@code{yyparse} should accept. The use of this macro is deprecated, and
11033is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
11034Conventions for Pure Parsers}.
11035@end deffn
11036
11037@deffn {Macro} YYRECOVERING
11038The expression @code{YYRECOVERING ()} yields 1 when the parser
11039is recovering from a syntax error, and 0 otherwise.
11040@xref{Action Features, ,Special Features for Use in Actions}.
11041@end deffn
11042
11043@deffn {Macro} YYSTACK_USE_ALLOCA
11044Macro used to control the use of @code{alloca} when the
11045deterministic parser in C needs to extend its stacks. If defined to 0,
11046the parser will use @code{malloc} to extend its stacks. If defined to
110471, the parser will use @code{alloca}. Values other than 0 and 1 are
11048reserved for future Bison extensions. If not defined,
11049@code{YYSTACK_USE_ALLOCA} defaults to 0.
11050
11051In the all-too-common case where your code may run on a host with a
11052limited stack and with unreliable stack-overflow checking, you should
11053set @code{YYMAXDEPTH} to a value that cannot possibly result in
11054unchecked stack overflow on any of your target hosts when
11055@code{alloca} is called. You can inspect the code that Bison
11056generates in order to determine the proper numeric values. This will
11057require some expertise in low-level implementation details.
11058@end deffn
11059
11060@deffn {Type} YYSTYPE
11061Data type of semantic values; @code{int} by default.
11062@xref{Value Type, ,Data Types of Semantic Values}.
11063@end deffn
11064
11065@node Glossary
11066@appendix Glossary
11067@cindex glossary
11068
11069@table @asis
11070@item Accepting State
11071A state whose only action is the accept action.
11072The accepting state is thus a consistent state.
11073@xref{Understanding,,}.
11074
11075@item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
11076Formal method of specifying context-free grammars originally proposed
11077by John Backus, and slightly improved by Peter Naur in his 1960-01-02
11078committee document contributing to what became the Algol 60 report.
11079@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11080
11081@item Consistent State
11082A state containing only one possible action.
11083@xref{Decl Summary,,lr.default-reductions}.
11084
11085@item Context-free grammars
11086Grammars specified as rules that can be applied regardless of context.
11087Thus, if there is a rule which says that an integer can be used as an
11088expression, integers are allowed @emph{anywhere} an expression is
11089permitted. @xref{Language and Grammar, ,Languages and Context-Free
11090Grammars}.
11091
11092@item Default Reduction
11093The reduction that a parser should perform if the current parser state
11094contains no other action for the lookahead token.
11095In permitted parser states, Bison declares the reduction with the
11096largest lookahead set to be the default reduction and removes that
11097lookahead set.
11098@xref{Decl Summary,,lr.default-reductions}.
11099
11100@item Dynamic allocation
11101Allocation of memory that occurs during execution, rather than at
11102compile time or on entry to a function.
11103
11104@item Empty string
11105Analogous to the empty set in set theory, the empty string is a
11106character string of length zero.
11107
11108@item Finite-state stack machine
11109A ``machine'' that has discrete states in which it is said to exist at
11110each instant in time. As input to the machine is processed, the
11111machine moves from state to state as specified by the logic of the
11112machine. In the case of the parser, the input is the language being
11113parsed, and the states correspond to various stages in the grammar
11114rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
11115
11116@item Generalized @acronym{LR} (@acronym{GLR})
11117A parsing algorithm that can handle all context-free grammars, including those
11118that are not @acronym{LR}(1). It resolves situations that Bison's
11119deterministic parsing
11120algorithm cannot by effectively splitting off multiple parsers, trying all
11121possible parsers, and discarding those that fail in the light of additional
11122right context. @xref{Generalized LR Parsing, ,Generalized
11123@acronym{LR} Parsing}.
11124
11125@item Grouping
11126A language construct that is (in general) grammatically divisible;
11127for example, `expression' or `declaration' in C@.
11128@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11129
11130@item @acronym{IELR}(1)
11131A minimal @acronym{LR}(1) parser table generation algorithm.
11132That is, given any context-free grammar, @acronym{IELR}(1) generates
11133parser tables with the full language recognition power of canonical
11134@acronym{LR}(1) but with nearly the same number of parser states as
11135@acronym{LALR}(1).
11136This reduction in parser states is often an order of magnitude.
11137More importantly, because canonical @acronym{LR}(1)'s extra parser
11138states may contain duplicate conflicts in the case of
11139non-@acronym{LR}(1) grammars, the number of conflicts for
11140@acronym{IELR}(1) is often an order of magnitude less as well.
11141This can significantly reduce the complexity of developing of a grammar.
11142@xref{Decl Summary,,lr.type}.
11143
11144@item Infix operator
11145An arithmetic operator that is placed between the operands on which it
11146performs some operation.
11147
11148@item Input stream
11149A continuous flow of data between devices or programs.
11150
11151@item Language construct
11152One of the typical usage schemas of the language. For example, one of
11153the constructs of the C language is the @code{if} statement.
11154@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11155
11156@item Left associativity
11157Operators having left associativity are analyzed from left to right:
11158@samp{a+b+c} first computes @samp{a+b} and then combines with
11159@samp{c}. @xref{Precedence, ,Operator Precedence}.
11160
11161@item Left recursion
11162A rule whose result symbol is also its first component symbol; for
11163example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
11164Rules}.
11165
11166@item Left-to-right parsing
11167Parsing a sentence of a language by analyzing it token by token from
11168left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
11169
11170@item Lexical analyzer (scanner)
11171A function that reads an input stream and returns tokens one by one.
11172@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
11173
11174@item Lexical tie-in
11175A flag, set by actions in the grammar rules, which alters the way
11176tokens are parsed. @xref{Lexical Tie-ins}.
11177
11178@item Literal string token
11179A token which consists of two or more fixed characters. @xref{Symbols}.
11180
11181@item Lookahead token
11182A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
11183Tokens}.
11184
11185@item @acronym{LALR}(1)
11186The class of context-free grammars that Bison (like most other parser
11187generators) can handle by default; a subset of @acronym{LR}(1).
11188@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
11189
11190@item @acronym{LR}(1)
11191The class of context-free grammars in which at most one token of
11192lookahead is needed to disambiguate the parsing of any piece of input.
11193
11194@item Nonterminal symbol
11195A grammar symbol standing for a grammatical construct that can
11196be expressed through rules in terms of smaller constructs; in other
11197words, a construct that is not a token. @xref{Symbols}.
11198
11199@item Parser
11200A function that recognizes valid sentences of a language by analyzing
11201the syntax structure of a set of tokens passed to it from a lexical
11202analyzer.
11203
11204@item Postfix operator
11205An arithmetic operator that is placed after the operands upon which it
11206performs some operation.
11207
11208@item Reduction
11209Replacing a string of nonterminals and/or terminals with a single
11210nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
11211Parser Algorithm}.
11212
11213@item Reentrant
11214A reentrant subprogram is a subprogram which can be in invoked any
11215number of times in parallel, without interference between the various
11216invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
11217
11218@item Reverse polish notation
11219A language in which all operators are postfix operators.
11220
11221@item Right recursion
11222A rule whose result symbol is also its last component symbol; for
11223example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
11224Rules}.
11225
11226@item Semantics
11227In computer languages, the semantics are specified by the actions
11228taken for each instance of the language, i.e., the meaning of
11229each statement. @xref{Semantics, ,Defining Language Semantics}.
11230
11231@item Shift
11232A parser is said to shift when it makes the choice of analyzing
11233further input from the stream rather than reducing immediately some
11234already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
11235
11236@item Single-character literal
11237A single character that is recognized and interpreted as is.
11238@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
11239
11240@item Start symbol
11241The nonterminal symbol that stands for a complete valid utterance in
11242the language being parsed. The start symbol is usually listed as the
11243first nonterminal symbol in a language specification.
11244@xref{Start Decl, ,The Start-Symbol}.
11245
11246@item Symbol table
11247A data structure where symbol names and associated data are stored
11248during parsing to allow for recognition and use of existing
11249information in repeated uses of a symbol. @xref{Multi-function Calc}.
11250
11251@item Syntax error
11252An error encountered during parsing of an input stream due to invalid
11253syntax. @xref{Error Recovery}.
11254
11255@item Token
11256A basic, grammatically indivisible unit of a language. The symbol
11257that describes a token in the grammar is a terminal symbol.
11258The input of the Bison parser is a stream of tokens which comes from
11259the lexical analyzer. @xref{Symbols}.
11260
11261@item Terminal symbol
11262A grammar symbol that has no rules in the grammar and therefore is
11263grammatically indivisible. The piece of text it represents is a token.
11264@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11265@end table
11266
11267@node Copying This Manual
11268@appendix Copying This Manual
11269@include fdl.texi
11270
11271@node Index
11272@unnumbered Index
11273
11274@printindex cp
11275
11276@bye
11277
11278@c Local Variables:
11279@c fill-column: 76
11280@c End:
11281
11282@c LocalWords: texinfo setfilename settitle setchapternewpage finalout texi FSF
11283@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex FSF's
11284@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry Naur
11285@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa Multi
11286@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc multi
11287@c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex defaultprec Donnelly Gotos
11288@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref yypush
11289@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex lr
11290@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge POSIX
11291@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG yypull
11292@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit nonfree
11293@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok rr
11294@c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln Stallman Destructor
11295@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym enum
11296@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof Lex
11297@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum DOTDOT
11298@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype Unary
11299@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs nonterminal
11300@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES reentrant
11301@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param yypstate
11302@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP subrange
11303@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword loc
11304@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH inline
11305@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype Lookahead
11306@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args Autoconf
11307@c LocalWords: infile ypp yxx outfile itemx tex leaderfill Troubleshouting sqrt
11308@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll lookahead
11309@c LocalWords: nbar yytext fst snd osplit ntwo strdup AST Troublereporting th
11310@c LocalWords: YYSTACK DVI fdl printindex IELR nondeterministic nonterminals ps
11311@c LocalWords: subexpressions declarator nondeferred config libintl postfix
11312@c LocalWords: preprocessor nonpositive unary nonnumeric typedef extern rhs
11313@c LocalWords: yytokentype filename destructor multicharacter nonnull EBCDIC
11314@c LocalWords: lvalue nonnegative XNUM CHR chr TAGLESS tagless stdout api TOK
11315@c LocalWords: destructors Reentrancy nonreentrant subgrammar nonassociative
11316@c LocalWords: deffnx namespace xml goto lalr ielr runtime lex yacc yyps env
11317@c LocalWords: yystate variadic Unshift NLS gettext po UTF Automake LOCALEDIR
11318@c LocalWords: YYENABLE bindtextdomain Makefile DEFS CPPFLAGS DBISON DeRemer
11319@c LocalWords: autoreconf Pennello multisets nondeterminism Generalised baz
11320@c LocalWords: redeclare automata Dparse localedir datadir XSLT midrule Wno
11321@c LocalWords: makefiles Graphviz multitable headitem hh basename Doxygen fno
11322@c LocalWords: doxygen ival sval deftypemethod deallocate pos deftypemethodx
11323@c LocalWords: Ctor defcv defcvx arg accessors arithmetics CPP ifndef CALCXX
11324@c LocalWords: lexer's calcxx bool LPAREN RPAREN deallocation cerrno climits
11325@c LocalWords: cstdlib Debian undef yywrap unput noyywrap nounput zA yyleng
11326@c LocalWords: errno strtol ERANGE str strerror iostream argc argv Javadoc
11327@c LocalWords: bytecode initializers superclass stype ASTNode autoboxing nls
11328@c LocalWords: toString deftypeivar deftypeivarx deftypeop YYParser strictfp
11329@c LocalWords: superclasses boolean getErrorVerbose setErrorVerbose deftypecv
11330@c LocalWords: getDebugStream setDebugStream getDebugLevel setDebugLevel url
11331@c LocalWords: bisonVersion deftypecvx bisonSkeleton getStartPos getEndPos
11332@c LocalWords: getLVal defvar YYFAIL deftypefn deftypefnx gotos msgfmt
11333@c LocalWords: subdirectory Solaris nonassociativity