]>
Commit | Line | Data |
---|---|---|
bfa74976 RS |
1 | \input texinfo @c -*-texinfo-*- |
2 | @comment %**start of header | |
3 | @setfilename bison.info | |
df1af54c JT |
4 | @include version.texi |
5 | @settitle Bison @value{VERSION} | |
bfa74976 RS |
6 | @setchapternewpage odd |
7 | ||
5378c3e7 | 8 | @finalout |
5378c3e7 | 9 | |
13863333 | 10 | @c SMALL BOOK version |
bfa74976 | 11 | @c This edition has been formatted so that you can format and print it in |
13863333 | 12 | @c the smallbook format. |
bfa74976 RS |
13 | @c @smallbook |
14 | ||
91d2c560 PE |
15 | @c Set following if you want to document %default-prec and %no-default-prec. |
16 | @c This feature is experimental and may change in future Bison versions. | |
17 | @c @set defaultprec | |
18 | ||
8c5b881d | 19 | @ifnotinfo |
bfa74976 RS |
20 | @syncodeindex fn cp |
21 | @syncodeindex vr cp | |
22 | @syncodeindex tp cp | |
8c5b881d | 23 | @end ifnotinfo |
bfa74976 RS |
24 | @ifinfo |
25 | @synindex fn cp | |
26 | @synindex vr cp | |
27 | @synindex tp cp | |
28 | @end ifinfo | |
29 | @comment %**end of header | |
30 | ||
fae437e8 | 31 | @copying |
bd773d73 | 32 | |
c827f760 PE |
33 | This manual is for @acronym{GNU} Bison (version @value{VERSION}, |
34 | @value{UPDATED}), the @acronym{GNU} parser generator. | |
fae437e8 | 35 | |
a06ea4aa | 36 | Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998, |
32c29292 | 37 | 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc. |
fae437e8 AD |
38 | |
39 | @quotation | |
40 | Permission is granted to copy, distribute and/or modify this document | |
c827f760 | 41 | under the terms of the @acronym{GNU} Free Documentation License, |
592fde95 | 42 | Version 1.2 or any later version published by the Free Software |
c827f760 PE |
43 | Foundation; with no Invariant Sections, with the Front-Cover texts |
44 | being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in | |
45 | (a) below. A copy of the license is included in the section entitled | |
46 | ``@acronym{GNU} Free Documentation License.'' | |
47 | ||
48 | (a) The @acronym{FSF}'s Back-Cover Text is: ``You have freedom to copy | |
49 | and modify this @acronym{GNU} Manual, like @acronym{GNU} software. | |
50 | Copies published by the Free Software Foundation raise funds for | |
51 | @acronym{GNU} development.'' | |
fae437e8 AD |
52 | @end quotation |
53 | @end copying | |
54 | ||
e62f1a89 | 55 | @dircategory Software development |
fae437e8 | 56 | @direntry |
c827f760 | 57 | * bison: (bison). @acronym{GNU} parser generator (Yacc replacement). |
fae437e8 | 58 | @end direntry |
bfa74976 | 59 | |
bfa74976 RS |
60 | @titlepage |
61 | @title Bison | |
c827f760 | 62 | @subtitle The Yacc-compatible Parser Generator |
df1af54c | 63 | @subtitle @value{UPDATED}, Bison Version @value{VERSION} |
bfa74976 RS |
64 | |
65 | @author by Charles Donnelly and Richard Stallman | |
66 | ||
67 | @page | |
68 | @vskip 0pt plus 1filll | |
fae437e8 | 69 | @insertcopying |
bfa74976 RS |
70 | @sp 2 |
71 | Published by the Free Software Foundation @* | |
0fb669f9 PE |
72 | 51 Franklin Street, Fifth Floor @* |
73 | Boston, MA 02110-1301 USA @* | |
9ecbd125 | 74 | Printed copies are available from the Free Software Foundation.@* |
c827f760 | 75 | @acronym{ISBN} 1-882114-44-2 |
bfa74976 RS |
76 | @sp 2 |
77 | Cover art by Etienne Suvasa. | |
78 | @end titlepage | |
d5796688 JT |
79 | |
80 | @contents | |
bfa74976 | 81 | |
342b8b6e AD |
82 | @ifnottex |
83 | @node Top | |
84 | @top Bison | |
fae437e8 | 85 | @insertcopying |
342b8b6e | 86 | @end ifnottex |
bfa74976 RS |
87 | |
88 | @menu | |
13863333 AD |
89 | * Introduction:: |
90 | * Conditions:: | |
c827f760 | 91 | * Copying:: The @acronym{GNU} General Public License says |
bfa74976 RS |
92 | how you can copy and share Bison |
93 | ||
94 | Tutorial sections: | |
95 | * Concepts:: Basic concepts for understanding Bison. | |
96 | * Examples:: Three simple explained examples of using Bison. | |
97 | ||
98 | Reference sections: | |
99 | * Grammar File:: Writing Bison declarations and rules. | |
100 | * Interface:: C-language interface to the parser function @code{yyparse}. | |
101 | * Algorithm:: How the Bison parser works at run-time. | |
102 | * Error Recovery:: Writing rules for error recovery. | |
103 | * Context Dependency:: What to do if your language syntax is too | |
104 | messy for Bison to handle straightforwardly. | |
ec3bc396 | 105 | * Debugging:: Understanding or debugging Bison parsers. |
bfa74976 | 106 | * Invocation:: How to run Bison (to produce the parser source file). |
12545799 AD |
107 | * C++ Language Interface:: Creating C++ parser objects. |
108 | * FAQ:: Frequently Asked Questions | |
bfa74976 RS |
109 | * Table of Symbols:: All the keywords of the Bison language are explained. |
110 | * Glossary:: Basic concepts are explained. | |
f2b5126e | 111 | * Copying This Manual:: License for copying this manual. |
bfa74976 RS |
112 | * Index:: Cross-references to the text. |
113 | ||
93dd49ab PE |
114 | @detailmenu |
115 | --- The Detailed Node Listing --- | |
bfa74976 RS |
116 | |
117 | The Concepts of Bison | |
118 | ||
119 | * Language and Grammar:: Languages and context-free grammars, | |
120 | as mathematical ideas. | |
121 | * Grammar in Bison:: How we represent grammars for Bison's sake. | |
122 | * Semantic Values:: Each token or syntactic grouping can have | |
123 | a semantic value (the value of an integer, | |
124 | the name of an identifier, etc.). | |
125 | * Semantic Actions:: Each rule can have an action containing C code. | |
99a9344e | 126 | * GLR Parsers:: Writing parsers for general context-free languages. |
93dd49ab | 127 | * Locations Overview:: Tracking Locations. |
bfa74976 RS |
128 | * Bison Parser:: What are Bison's input and output, |
129 | how is the output used? | |
130 | * Stages:: Stages in writing and running Bison grammars. | |
131 | * Grammar Layout:: Overall structure of a Bison grammar file. | |
132 | ||
fa7e68c3 PE |
133 | Writing @acronym{GLR} Parsers |
134 | ||
32c29292 JD |
135 | * Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars. |
136 | * Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities. | |
137 | * GLR Semantic Actions:: Deferred semantic actions have special concerns. | |
138 | * Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler. | |
fa7e68c3 | 139 | |
bfa74976 RS |
140 | Examples |
141 | ||
142 | * RPN Calc:: Reverse polish notation calculator; | |
143 | a first example with no operator precedence. | |
144 | * Infix Calc:: Infix (algebraic) notation calculator. | |
145 | Operator precedence is introduced. | |
146 | * Simple Error Recovery:: Continuing after syntax errors. | |
342b8b6e | 147 | * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$. |
93dd49ab PE |
148 | * Multi-function Calc:: Calculator with memory and trig functions. |
149 | It uses multiple data-types for semantic values. | |
bfa74976 RS |
150 | * Exercises:: Ideas for improving the multi-function calculator. |
151 | ||
152 | Reverse Polish Notation Calculator | |
153 | ||
75f5aaea | 154 | * Decls: Rpcalc Decls. Prologue (declarations) for rpcalc. |
bfa74976 RS |
155 | * Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation. |
156 | * Lexer: Rpcalc Lexer. The lexical analyzer. | |
157 | * Main: Rpcalc Main. The controlling function. | |
158 | * Error: Rpcalc Error. The error reporting function. | |
159 | * Gen: Rpcalc Gen. Running Bison on the grammar file. | |
160 | * Comp: Rpcalc Compile. Run the C compiler on the output code. | |
161 | ||
162 | Grammar Rules for @code{rpcalc} | |
163 | ||
13863333 AD |
164 | * Rpcalc Input:: |
165 | * Rpcalc Line:: | |
166 | * Rpcalc Expr:: | |
bfa74976 | 167 | |
342b8b6e AD |
168 | Location Tracking Calculator: @code{ltcalc} |
169 | ||
170 | * Decls: Ltcalc Decls. Bison and C declarations for ltcalc. | |
171 | * Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations. | |
172 | * Lexer: Ltcalc Lexer. The lexical analyzer. | |
173 | ||
bfa74976 RS |
174 | Multi-Function Calculator: @code{mfcalc} |
175 | ||
176 | * Decl: Mfcalc Decl. Bison declarations for multi-function calculator. | |
177 | * Rules: Mfcalc Rules. Grammar rules for the calculator. | |
178 | * Symtab: Mfcalc Symtab. Symbol table management subroutines. | |
179 | ||
180 | Bison Grammar Files | |
181 | ||
182 | * Grammar Outline:: Overall layout of the grammar file. | |
183 | * Symbols:: Terminal and nonterminal symbols. | |
184 | * Rules:: How to write grammar rules. | |
185 | * Recursion:: Writing recursive rules. | |
186 | * Semantics:: Semantic values and actions. | |
93dd49ab | 187 | * Locations:: Locations and actions. |
bfa74976 RS |
188 | * Declarations:: All kinds of Bison declarations are described here. |
189 | * Multiple Parsers:: Putting more than one Bison parser in one program. | |
190 | ||
191 | Outline of a Bison Grammar | |
192 | ||
93dd49ab | 193 | * Prologue:: Syntax and usage of the prologue. |
bfa74976 RS |
194 | * Bison Declarations:: Syntax and usage of the Bison declarations section. |
195 | * Grammar Rules:: Syntax and usage of the grammar rules section. | |
93dd49ab | 196 | * Epilogue:: Syntax and usage of the epilogue. |
bfa74976 RS |
197 | |
198 | Defining Language Semantics | |
199 | ||
200 | * Value Type:: Specifying one data type for all semantic values. | |
201 | * Multiple Types:: Specifying several alternative data types. | |
202 | * Actions:: An action is the semantic definition of a grammar rule. | |
203 | * Action Types:: Specifying data types for actions to operate on. | |
204 | * Mid-Rule Actions:: Most actions go at the end of a rule. | |
205 | This says when, why and how to use the exceptional | |
206 | action in the middle of a rule. | |
207 | ||
93dd49ab PE |
208 | Tracking Locations |
209 | ||
210 | * Location Type:: Specifying a data type for locations. | |
211 | * Actions and Locations:: Using locations in actions. | |
212 | * Location Default Action:: Defining a general way to compute locations. | |
213 | ||
bfa74976 RS |
214 | Bison Declarations |
215 | ||
b50d2359 | 216 | * Require Decl:: Requiring a Bison version. |
bfa74976 RS |
217 | * Token Decl:: Declaring terminal symbols. |
218 | * Precedence Decl:: Declaring terminals with precedence and associativity. | |
219 | * Union Decl:: Declaring the set of all semantic value types. | |
220 | * Type Decl:: Declaring the choice of type for a nonterminal symbol. | |
18d192f0 | 221 | * Initial Action Decl:: Code run before parsing starts. |
72f889cc | 222 | * Destructor Decl:: Declaring how symbols are freed. |
d6328241 | 223 | * Expect Decl:: Suppressing warnings about parsing conflicts. |
bfa74976 RS |
224 | * Start Decl:: Specifying the start symbol. |
225 | * Pure Decl:: Requesting a reentrant parser. | |
226 | * Decl Summary:: Table of all Bison declarations. | |
227 | ||
228 | Parser C-Language Interface | |
229 | ||
230 | * Parser Function:: How to call @code{yyparse} and what it returns. | |
13863333 | 231 | * Lexical:: You must supply a function @code{yylex} |
bfa74976 RS |
232 | which reads tokens. |
233 | * Error Reporting:: You must supply a function @code{yyerror}. | |
234 | * Action Features:: Special features for use in actions. | |
f7ab6a50 PE |
235 | * Internationalization:: How to let the parser speak in the user's |
236 | native language. | |
bfa74976 RS |
237 | |
238 | The Lexical Analyzer Function @code{yylex} | |
239 | ||
240 | * Calling Convention:: How @code{yyparse} calls @code{yylex}. | |
241 | * Token Values:: How @code{yylex} must return the semantic value | |
242 | of the token it has read. | |
95923bd6 | 243 | * Token Locations:: How @code{yylex} must return the text location |
bfa74976 | 244 | (line number, etc.) of the token, if the |
93dd49ab | 245 | actions want that. |
bfa74976 RS |
246 | * Pure Calling:: How the calling convention differs |
247 | in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). | |
248 | ||
13863333 | 249 | The Bison Parser Algorithm |
bfa74976 | 250 | |
742e4900 | 251 | * Lookahead:: Parser looks one token ahead when deciding what to do. |
bfa74976 RS |
252 | * Shift/Reduce:: Conflicts: when either shifting or reduction is valid. |
253 | * Precedence:: Operator precedence works by resolving conflicts. | |
254 | * Contextual Precedence:: When an operator's precedence depends on context. | |
255 | * Parser States:: The parser is a finite-state-machine with stack. | |
256 | * Reduce/Reduce:: When two rules are applicable in the same situation. | |
257 | * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified. | |
676385e2 | 258 | * Generalized LR Parsing:: Parsing arbitrary context-free grammars. |
1a059451 | 259 | * Memory Management:: What happens when memory is exhausted. How to avoid it. |
bfa74976 RS |
260 | |
261 | Operator Precedence | |
262 | ||
263 | * Why Precedence:: An example showing why precedence is needed. | |
264 | * Using Precedence:: How to specify precedence in Bison grammars. | |
265 | * Precedence Examples:: How these features are used in the previous example. | |
266 | * How Precedence:: How they work. | |
267 | ||
268 | Handling Context Dependencies | |
269 | ||
270 | * Semantic Tokens:: Token parsing can depend on the semantic context. | |
271 | * Lexical Tie-ins:: Token parsing can depend on the syntactic context. | |
272 | * Tie-in Recovery:: Lexical tie-ins have implications for how | |
273 | error recovery rules must be written. | |
274 | ||
93dd49ab | 275 | Debugging Your Parser |
ec3bc396 AD |
276 | |
277 | * Understanding:: Understanding the structure of your parser. | |
278 | * Tracing:: Tracing the execution of your parser. | |
279 | ||
bfa74976 RS |
280 | Invoking Bison |
281 | ||
13863333 | 282 | * Bison Options:: All the options described in detail, |
c827f760 | 283 | in alphabetical order by short options. |
bfa74976 | 284 | * Option Cross Key:: Alphabetical list of long options. |
93dd49ab | 285 | * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}. |
f2b5126e | 286 | |
12545799 AD |
287 | C++ Language Interface |
288 | ||
289 | * C++ Parsers:: The interface to generate C++ parser classes | |
290 | * A Complete C++ Example:: Demonstrating their use | |
291 | ||
292 | C++ Parsers | |
293 | ||
294 | * C++ Bison Interface:: Asking for C++ parser generation | |
295 | * C++ Semantic Values:: %union vs. C++ | |
296 | * C++ Location Values:: The position and location classes | |
297 | * C++ Parser Interface:: Instantiating and running the parser | |
298 | * C++ Scanner Interface:: Exchanges between yylex and parse | |
299 | ||
300 | A Complete C++ Example | |
301 | ||
302 | * Calc++ --- C++ Calculator:: The specifications | |
303 | * Calc++ Parsing Driver:: An active parsing context | |
304 | * Calc++ Parser:: A parser class | |
305 | * Calc++ Scanner:: A pure C++ Flex scanner | |
306 | * Calc++ Top Level:: Conducting the band | |
307 | ||
d1a1114f AD |
308 | Frequently Asked Questions |
309 | ||
1a059451 | 310 | * Memory Exhausted:: Breaking the Stack Limits |
e64fec0a | 311 | * How Can I Reset the Parser:: @code{yyparse} Keeps some State |
fef4cb51 | 312 | * Strings are Destroyed:: @code{yylval} Loses Track of Strings |
2fa09258 | 313 | * Implementing Gotos/Loops:: Control Flow in the Calculator |
ed2e6384 | 314 | * Multiple start-symbols:: Factoring closely related grammars |
55ba27be AD |
315 | * Secure? Conform?:: Is Bison @acronym{POSIX} safe? |
316 | * I can't build Bison:: Troubleshooting | |
317 | * Where can I find help?:: Troubleshouting | |
318 | * Bug Reports:: Troublereporting | |
319 | * Other Languages:: Parsers in Java and others | |
320 | * Beta Testing:: Experimenting development versions | |
321 | * Mailing Lists:: Meeting other Bison users | |
d1a1114f | 322 | |
f2b5126e PB |
323 | Copying This Manual |
324 | ||
325 | * GNU Free Documentation License:: License for copying this manual. | |
326 | ||
342b8b6e | 327 | @end detailmenu |
bfa74976 RS |
328 | @end menu |
329 | ||
342b8b6e | 330 | @node Introduction |
bfa74976 RS |
331 | @unnumbered Introduction |
332 | @cindex introduction | |
333 | ||
6077da58 PE |
334 | @dfn{Bison} is a general-purpose parser generator that converts an |
335 | annotated context-free grammar into an @acronym{LALR}(1) or | |
336 | @acronym{GLR} parser for that grammar. Once you are proficient with | |
1e137b71 | 337 | Bison, you can use it to develop a wide range of language parsers, from those |
bfa74976 RS |
338 | used in simple desk calculators to complex programming languages. |
339 | ||
340 | Bison is upward compatible with Yacc: all properly-written Yacc grammars | |
341 | ought to work with Bison with no change. Anyone familiar with Yacc | |
342 | should be able to use Bison with little trouble. You need to be fluent in | |
1e137b71 | 343 | C or C++ programming in order to use Bison or to understand this manual. |
bfa74976 RS |
344 | |
345 | We begin with tutorial chapters that explain the basic concepts of using | |
346 | Bison and show three explained examples, each building on the last. If you | |
347 | don't know Bison or Yacc, start by reading these chapters. Reference | |
348 | chapters follow which describe specific aspects of Bison in detail. | |
349 | ||
931c7513 RS |
350 | Bison was written primarily by Robert Corbett; Richard Stallman made it |
351 | Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added | |
14ded682 | 352 | multi-character string literals and other features. |
931c7513 | 353 | |
df1af54c | 354 | This edition corresponds to version @value{VERSION} of Bison. |
bfa74976 | 355 | |
342b8b6e | 356 | @node Conditions |
bfa74976 RS |
357 | @unnumbered Conditions for Using Bison |
358 | ||
193d7c70 PE |
359 | The distribution terms for Bison-generated parsers permit using the |
360 | parsers in nonfree programs. Before Bison version 2.2, these extra | |
361 | permissions applied only when Bison was generating @acronym{LALR}(1) | |
362 | parsers in C@. And before Bison version 1.24, Bison-generated | |
262aa8dd | 363 | parsers could be used only in programs that were free software. |
a31239f1 | 364 | |
c827f760 PE |
365 | The other @acronym{GNU} programming tools, such as the @acronym{GNU} C |
366 | compiler, have never | |
9ecbd125 | 367 | had such a requirement. They could always be used for nonfree |
a31239f1 RS |
368 | software. The reason Bison was different was not due to a special |
369 | policy decision; it resulted from applying the usual General Public | |
370 | License to all of the Bison source code. | |
371 | ||
372 | The output of the Bison utility---the Bison parser file---contains a | |
373 | verbatim copy of a sizable piece of Bison, which is the code for the | |
193d7c70 PE |
374 | parser's implementation. (The actions from your grammar are inserted |
375 | into this implementation at one point, but most of the rest of the | |
376 | implementation is not changed.) When we applied the @acronym{GPL} | |
377 | terms to the skeleton code for the parser's implementation, | |
a31239f1 RS |
378 | the effect was to restrict the use of Bison output to free software. |
379 | ||
380 | We didn't change the terms because of sympathy for people who want to | |
381 | make software proprietary. @strong{Software should be free.} But we | |
382 | concluded that limiting Bison's use to free software was doing little to | |
383 | encourage people to make other software free. So we decided to make the | |
384 | practical conditions for using Bison match the practical conditions for | |
c827f760 | 385 | using the other @acronym{GNU} tools. |
bfa74976 | 386 | |
193d7c70 PE |
387 | This exception applies when Bison is generating code for a parser. |
388 | You can tell whether the exception applies to a Bison output file by | |
389 | inspecting the file for text beginning with ``As a special | |
390 | exception@dots{}''. The text spells out the exact terms of the | |
391 | exception. | |
262aa8dd | 392 | |
c67a198d | 393 | @include gpl.texi |
bfa74976 | 394 | |
342b8b6e | 395 | @node Concepts |
bfa74976 RS |
396 | @chapter The Concepts of Bison |
397 | ||
398 | This chapter introduces many of the basic concepts without which the | |
399 | details of Bison will not make sense. If you do not already know how to | |
400 | use Bison or Yacc, we suggest you start by reading this chapter carefully. | |
401 | ||
402 | @menu | |
403 | * Language and Grammar:: Languages and context-free grammars, | |
404 | as mathematical ideas. | |
405 | * Grammar in Bison:: How we represent grammars for Bison's sake. | |
406 | * Semantic Values:: Each token or syntactic grouping can have | |
407 | a semantic value (the value of an integer, | |
408 | the name of an identifier, etc.). | |
409 | * Semantic Actions:: Each rule can have an action containing C code. | |
99a9344e | 410 | * GLR Parsers:: Writing parsers for general context-free languages. |
847bf1f5 | 411 | * Locations Overview:: Tracking Locations. |
bfa74976 RS |
412 | * Bison Parser:: What are Bison's input and output, |
413 | how is the output used? | |
414 | * Stages:: Stages in writing and running Bison grammars. | |
415 | * Grammar Layout:: Overall structure of a Bison grammar file. | |
416 | @end menu | |
417 | ||
342b8b6e | 418 | @node Language and Grammar |
bfa74976 RS |
419 | @section Languages and Context-Free Grammars |
420 | ||
bfa74976 RS |
421 | @cindex context-free grammar |
422 | @cindex grammar, context-free | |
423 | In order for Bison to parse a language, it must be described by a | |
424 | @dfn{context-free grammar}. This means that you specify one or more | |
425 | @dfn{syntactic groupings} and give rules for constructing them from their | |
426 | parts. For example, in the C language, one kind of grouping is called an | |
427 | `expression'. One rule for making an expression might be, ``An expression | |
428 | can be made of a minus sign and another expression''. Another would be, | |
429 | ``An expression can be an integer''. As you can see, rules are often | |
430 | recursive, but there must be at least one rule which leads out of the | |
431 | recursion. | |
432 | ||
c827f760 | 433 | @cindex @acronym{BNF} |
bfa74976 RS |
434 | @cindex Backus-Naur form |
435 | The most common formal system for presenting such rules for humans to read | |
c827f760 PE |
436 | is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in |
437 | order to specify the language Algol 60. Any grammar expressed in | |
438 | @acronym{BNF} is a context-free grammar. The input to Bison is | |
439 | essentially machine-readable @acronym{BNF}. | |
bfa74976 | 440 | |
c827f760 PE |
441 | @cindex @acronym{LALR}(1) grammars |
442 | @cindex @acronym{LR}(1) grammars | |
676385e2 PH |
443 | There are various important subclasses of context-free grammar. Although it |
444 | can handle almost all context-free grammars, Bison is optimized for what | |
c827f760 | 445 | are called @acronym{LALR}(1) grammars. |
676385e2 | 446 | In brief, in these grammars, it must be possible to |
bfa74976 | 447 | tell how to parse any portion of an input string with just a single |
742e4900 | 448 | token of lookahead. Strictly speaking, that is a description of an |
c827f760 PE |
449 | @acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional |
450 | restrictions that are | |
bfa74976 | 451 | hard to explain simply; but it is rare in actual practice to find an |
c827f760 PE |
452 | @acronym{LR}(1) grammar that fails to be @acronym{LALR}(1). |
453 | @xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for | |
454 | more information on this. | |
bfa74976 | 455 | |
c827f760 PE |
456 | @cindex @acronym{GLR} parsing |
457 | @cindex generalized @acronym{LR} (@acronym{GLR}) parsing | |
676385e2 | 458 | @cindex ambiguous grammars |
9d9b8b70 | 459 | @cindex nondeterministic parsing |
9501dc6e AD |
460 | |
461 | Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning | |
462 | roughly that the next grammar rule to apply at any point in the input is | |
463 | uniquely determined by the preceding input and a fixed, finite portion | |
742e4900 | 464 | (called a @dfn{lookahead}) of the remaining input. A context-free |
9501dc6e | 465 | grammar can be @dfn{ambiguous}, meaning that there are multiple ways to |
e4f85c39 | 466 | apply the grammar rules to get the same inputs. Even unambiguous |
9d9b8b70 | 467 | grammars can be @dfn{nondeterministic}, meaning that no fixed |
742e4900 | 468 | lookahead always suffices to determine the next grammar rule to apply. |
9501dc6e AD |
469 | With the proper declarations, Bison is also able to parse these more |
470 | general context-free grammars, using a technique known as @acronym{GLR} | |
471 | parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers | |
472 | are able to handle any context-free grammar for which the number of | |
473 | possible parses of any given string is finite. | |
676385e2 | 474 | |
bfa74976 RS |
475 | @cindex symbols (abstract) |
476 | @cindex token | |
477 | @cindex syntactic grouping | |
478 | @cindex grouping, syntactic | |
9501dc6e AD |
479 | In the formal grammatical rules for a language, each kind of syntactic |
480 | unit or grouping is named by a @dfn{symbol}. Those which are built by | |
481 | grouping smaller constructs according to grammatical rules are called | |
bfa74976 RS |
482 | @dfn{nonterminal symbols}; those which can't be subdivided are called |
483 | @dfn{terminal symbols} or @dfn{token types}. We call a piece of input | |
484 | corresponding to a single terminal symbol a @dfn{token}, and a piece | |
e0c471a9 | 485 | corresponding to a single nonterminal symbol a @dfn{grouping}. |
bfa74976 RS |
486 | |
487 | We can use the C language as an example of what symbols, terminal and | |
9501dc6e AD |
488 | nonterminal, mean. The tokens of C are identifiers, constants (numeric |
489 | and string), and the various keywords, arithmetic operators and | |
490 | punctuation marks. So the terminal symbols of a grammar for C include | |
491 | `identifier', `number', `string', plus one symbol for each keyword, | |
492 | operator or punctuation mark: `if', `return', `const', `static', `int', | |
493 | `char', `plus-sign', `open-brace', `close-brace', `comma' and many more. | |
494 | (These tokens can be subdivided into characters, but that is a matter of | |
bfa74976 RS |
495 | lexicography, not grammar.) |
496 | ||
497 | Here is a simple C function subdivided into tokens: | |
498 | ||
9edcd895 AD |
499 | @ifinfo |
500 | @example | |
501 | int /* @r{keyword `int'} */ | |
14d4662b | 502 | square (int x) /* @r{identifier, open-paren, keyword `int',} |
9edcd895 AD |
503 | @r{identifier, close-paren} */ |
504 | @{ /* @r{open-brace} */ | |
aa08666d AD |
505 | return x * x; /* @r{keyword `return', identifier, asterisk,} |
506 | @r{identifier, semicolon} */ | |
9edcd895 AD |
507 | @} /* @r{close-brace} */ |
508 | @end example | |
509 | @end ifinfo | |
510 | @ifnotinfo | |
bfa74976 RS |
511 | @example |
512 | int /* @r{keyword `int'} */ | |
14d4662b | 513 | square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */ |
bfa74976 | 514 | @{ /* @r{open-brace} */ |
9edcd895 | 515 | return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */ |
bfa74976 RS |
516 | @} /* @r{close-brace} */ |
517 | @end example | |
9edcd895 | 518 | @end ifnotinfo |
bfa74976 RS |
519 | |
520 | The syntactic groupings of C include the expression, the statement, the | |
521 | declaration, and the function definition. These are represented in the | |
522 | grammar of C by nonterminal symbols `expression', `statement', | |
523 | `declaration' and `function definition'. The full grammar uses dozens of | |
524 | additional language constructs, each with its own nonterminal symbol, in | |
525 | order to express the meanings of these four. The example above is a | |
526 | function definition; it contains one declaration, and one statement. In | |
527 | the statement, each @samp{x} is an expression and so is @samp{x * x}. | |
528 | ||
529 | Each nonterminal symbol must have grammatical rules showing how it is made | |
530 | out of simpler constructs. For example, one kind of C statement is the | |
531 | @code{return} statement; this would be described with a grammar rule which | |
532 | reads informally as follows: | |
533 | ||
534 | @quotation | |
535 | A `statement' can be made of a `return' keyword, an `expression' and a | |
536 | `semicolon'. | |
537 | @end quotation | |
538 | ||
539 | @noindent | |
540 | There would be many other rules for `statement', one for each kind of | |
541 | statement in C. | |
542 | ||
543 | @cindex start symbol | |
544 | One nonterminal symbol must be distinguished as the special one which | |
545 | defines a complete utterance in the language. It is called the @dfn{start | |
546 | symbol}. In a compiler, this means a complete input program. In the C | |
547 | language, the nonterminal symbol `sequence of definitions and declarations' | |
548 | plays this role. | |
549 | ||
550 | For example, @samp{1 + 2} is a valid C expression---a valid part of a C | |
551 | program---but it is not valid as an @emph{entire} C program. In the | |
552 | context-free grammar of C, this follows from the fact that `expression' is | |
553 | not the start symbol. | |
554 | ||
555 | The Bison parser reads a sequence of tokens as its input, and groups the | |
556 | tokens using the grammar rules. If the input is valid, the end result is | |
557 | that the entire token sequence reduces to a single grouping whose symbol is | |
558 | the grammar's start symbol. If we use a grammar for C, the entire input | |
559 | must be a `sequence of definitions and declarations'. If not, the parser | |
560 | reports a syntax error. | |
561 | ||
342b8b6e | 562 | @node Grammar in Bison |
bfa74976 RS |
563 | @section From Formal Rules to Bison Input |
564 | @cindex Bison grammar | |
565 | @cindex grammar, Bison | |
566 | @cindex formal grammar | |
567 | ||
568 | A formal grammar is a mathematical construct. To define the language | |
569 | for Bison, you must write a file expressing the grammar in Bison syntax: | |
570 | a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}. | |
571 | ||
572 | A nonterminal symbol in the formal grammar is represented in Bison input | |
c827f760 | 573 | as an identifier, like an identifier in C@. By convention, it should be |
bfa74976 RS |
574 | in lower case, such as @code{expr}, @code{stmt} or @code{declaration}. |
575 | ||
576 | The Bison representation for a terminal symbol is also called a @dfn{token | |
577 | type}. Token types as well can be represented as C-like identifiers. By | |
578 | convention, these identifiers should be upper case to distinguish them from | |
579 | nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or | |
580 | @code{RETURN}. A terminal symbol that stands for a particular keyword in | |
581 | the language should be named after that keyword converted to upper case. | |
582 | The terminal symbol @code{error} is reserved for error recovery. | |
931c7513 | 583 | @xref{Symbols}. |
bfa74976 RS |
584 | |
585 | A terminal symbol can also be represented as a character literal, just like | |
586 | a C character constant. You should do this whenever a token is just a | |
587 | single character (parenthesis, plus-sign, etc.): use that same character in | |
588 | a literal as the terminal symbol for that token. | |
589 | ||
931c7513 RS |
590 | A third way to represent a terminal symbol is with a C string constant |
591 | containing several characters. @xref{Symbols}, for more information. | |
592 | ||
bfa74976 RS |
593 | The grammar rules also have an expression in Bison syntax. For example, |
594 | here is the Bison rule for a C @code{return} statement. The semicolon in | |
595 | quotes is a literal character token, representing part of the C syntax for | |
596 | the statement; the naked semicolon, and the colon, are Bison punctuation | |
597 | used in every rule. | |
598 | ||
599 | @example | |
600 | stmt: RETURN expr ';' | |
601 | ; | |
602 | @end example | |
603 | ||
604 | @noindent | |
605 | @xref{Rules, ,Syntax of Grammar Rules}. | |
606 | ||
342b8b6e | 607 | @node Semantic Values |
bfa74976 RS |
608 | @section Semantic Values |
609 | @cindex semantic value | |
610 | @cindex value, semantic | |
611 | ||
612 | A formal grammar selects tokens only by their classifications: for example, | |
613 | if a rule mentions the terminal symbol `integer constant', it means that | |
614 | @emph{any} integer constant is grammatically valid in that position. The | |
615 | precise value of the constant is irrelevant to how to parse the input: if | |
616 | @samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally | |
e0c471a9 | 617 | grammatical. |
bfa74976 RS |
618 | |
619 | But the precise value is very important for what the input means once it is | |
620 | parsed. A compiler is useless if it fails to distinguish between 4, 1 and | |
621 | 3989 as constants in the program! Therefore, each token in a Bison grammar | |
c827f760 PE |
622 | has both a token type and a @dfn{semantic value}. @xref{Semantics, |
623 | ,Defining Language Semantics}, | |
bfa74976 RS |
624 | for details. |
625 | ||
626 | The token type is a terminal symbol defined in the grammar, such as | |
627 | @code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything | |
628 | you need to know to decide where the token may validly appear and how to | |
629 | group it with other tokens. The grammar rules know nothing about tokens | |
e0c471a9 | 630 | except their types. |
bfa74976 RS |
631 | |
632 | The semantic value has all the rest of the information about the | |
633 | meaning of the token, such as the value of an integer, or the name of an | |
634 | identifier. (A token such as @code{','} which is just punctuation doesn't | |
635 | need to have any semantic value.) | |
636 | ||
637 | For example, an input token might be classified as token type | |
638 | @code{INTEGER} and have the semantic value 4. Another input token might | |
639 | have the same token type @code{INTEGER} but value 3989. When a grammar | |
640 | rule says that @code{INTEGER} is allowed, either of these tokens is | |
641 | acceptable because each is an @code{INTEGER}. When the parser accepts the | |
642 | token, it keeps track of the token's semantic value. | |
643 | ||
644 | Each grouping can also have a semantic value as well as its nonterminal | |
645 | symbol. For example, in a calculator, an expression typically has a | |
646 | semantic value that is a number. In a compiler for a programming | |
647 | language, an expression typically has a semantic value that is a tree | |
648 | structure describing the meaning of the expression. | |
649 | ||
342b8b6e | 650 | @node Semantic Actions |
bfa74976 RS |
651 | @section Semantic Actions |
652 | @cindex semantic actions | |
653 | @cindex actions, semantic | |
654 | ||
655 | In order to be useful, a program must do more than parse input; it must | |
656 | also produce some output based on the input. In a Bison grammar, a grammar | |
657 | rule can have an @dfn{action} made up of C statements. Each time the | |
658 | parser recognizes a match for that rule, the action is executed. | |
659 | @xref{Actions}. | |
13863333 | 660 | |
bfa74976 RS |
661 | Most of the time, the purpose of an action is to compute the semantic value |
662 | of the whole construct from the semantic values of its parts. For example, | |
663 | suppose we have a rule which says an expression can be the sum of two | |
664 | expressions. When the parser recognizes such a sum, each of the | |
665 | subexpressions has a semantic value which describes how it was built up. | |
666 | The action for this rule should create a similar sort of value for the | |
667 | newly recognized larger expression. | |
668 | ||
669 | For example, here is a rule that says an expression can be the sum of | |
670 | two subexpressions: | |
671 | ||
672 | @example | |
673 | expr: expr '+' expr @{ $$ = $1 + $3; @} | |
674 | ; | |
675 | @end example | |
676 | ||
677 | @noindent | |
678 | The action says how to produce the semantic value of the sum expression | |
679 | from the values of the two subexpressions. | |
680 | ||
676385e2 | 681 | @node GLR Parsers |
c827f760 PE |
682 | @section Writing @acronym{GLR} Parsers |
683 | @cindex @acronym{GLR} parsing | |
684 | @cindex generalized @acronym{LR} (@acronym{GLR}) parsing | |
676385e2 PH |
685 | @findex %glr-parser |
686 | @cindex conflicts | |
687 | @cindex shift/reduce conflicts | |
fa7e68c3 | 688 | @cindex reduce/reduce conflicts |
676385e2 | 689 | |
fa7e68c3 | 690 | In some grammars, Bison's standard |
9501dc6e AD |
691 | @acronym{LALR}(1) parsing algorithm cannot decide whether to apply a |
692 | certain grammar rule at a given point. That is, it may not be able to | |
693 | decide (on the basis of the input read so far) which of two possible | |
694 | reductions (applications of a grammar rule) applies, or whether to apply | |
695 | a reduction or read more of the input and apply a reduction later in the | |
696 | input. These are known respectively as @dfn{reduce/reduce} conflicts | |
697 | (@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts | |
698 | (@pxref{Shift/Reduce}). | |
699 | ||
700 | To use a grammar that is not easily modified to be @acronym{LALR}(1), a | |
701 | more general parsing algorithm is sometimes necessary. If you include | |
676385e2 | 702 | @code{%glr-parser} among the Bison declarations in your file |
fa7e68c3 | 703 | (@pxref{Grammar Outline}), the result is a Generalized @acronym{LR} |
9501dc6e AD |
704 | (@acronym{GLR}) parser. These parsers handle Bison grammars that |
705 | contain no unresolved conflicts (i.e., after applying precedence | |
706 | declarations) identically to @acronym{LALR}(1) parsers. However, when | |
707 | faced with unresolved shift/reduce and reduce/reduce conflicts, | |
708 | @acronym{GLR} parsers use the simple expedient of doing both, | |
709 | effectively cloning the parser to follow both possibilities. Each of | |
710 | the resulting parsers can again split, so that at any given time, there | |
711 | can be any number of possible parses being explored. The parsers | |
676385e2 PH |
712 | proceed in lockstep; that is, all of them consume (shift) a given input |
713 | symbol before any of them proceed to the next. Each of the cloned | |
714 | parsers eventually meets one of two possible fates: either it runs into | |
715 | a parsing error, in which case it simply vanishes, or it merges with | |
716 | another parser, because the two of them have reduced the input to an | |
717 | identical set of symbols. | |
718 | ||
719 | During the time that there are multiple parsers, semantic actions are | |
720 | recorded, but not performed. When a parser disappears, its recorded | |
721 | semantic actions disappear as well, and are never performed. When a | |
722 | reduction makes two parsers identical, causing them to merge, Bison | |
723 | records both sets of semantic actions. Whenever the last two parsers | |
724 | merge, reverting to the single-parser case, Bison resolves all the | |
725 | outstanding actions either by precedences given to the grammar rules | |
726 | involved, or by performing both actions, and then calling a designated | |
727 | user-defined function on the resulting values to produce an arbitrary | |
728 | merged result. | |
729 | ||
fa7e68c3 | 730 | @menu |
32c29292 JD |
731 | * Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars. |
732 | * Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities. | |
733 | * GLR Semantic Actions:: Deferred semantic actions have special concerns. | |
734 | * Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler. | |
fa7e68c3 PE |
735 | @end menu |
736 | ||
737 | @node Simple GLR Parsers | |
738 | @subsection Using @acronym{GLR} on Unambiguous Grammars | |
739 | @cindex @acronym{GLR} parsing, unambiguous grammars | |
740 | @cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars | |
741 | @findex %glr-parser | |
742 | @findex %expect-rr | |
743 | @cindex conflicts | |
744 | @cindex reduce/reduce conflicts | |
745 | @cindex shift/reduce conflicts | |
746 | ||
747 | In the simplest cases, you can use the @acronym{GLR} algorithm | |
748 | to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1). | |
742e4900 | 749 | Such grammars typically require more than one symbol of lookahead, |
fa7e68c3 PE |
750 | or (in rare cases) fall into the category of grammars in which the |
751 | @acronym{LALR}(1) algorithm throws away too much information (they are in | |
752 | @acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}). | |
753 | ||
754 | Consider a problem that | |
755 | arises in the declaration of enumerated and subrange types in the | |
756 | programming language Pascal. Here are some examples: | |
757 | ||
758 | @example | |
759 | type subrange = lo .. hi; | |
760 | type enum = (a, b, c); | |
761 | @end example | |
762 | ||
763 | @noindent | |
764 | The original language standard allows only numeric | |
765 | literals and constant identifiers for the subrange bounds (@samp{lo} | |
766 | and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC} | |
767 | 10206) and many other | |
768 | Pascal implementations allow arbitrary expressions there. This gives | |
769 | rise to the following situation, containing a superfluous pair of | |
770 | parentheses: | |
771 | ||
772 | @example | |
773 | type subrange = (a) .. b; | |
774 | @end example | |
775 | ||
776 | @noindent | |
777 | Compare this to the following declaration of an enumerated | |
778 | type with only one value: | |
779 | ||
780 | @example | |
781 | type enum = (a); | |
782 | @end example | |
783 | ||
784 | @noindent | |
785 | (These declarations are contrived, but they are syntactically | |
786 | valid, and more-complicated cases can come up in practical programs.) | |
787 | ||
788 | These two declarations look identical until the @samp{..} token. | |
742e4900 | 789 | With normal @acronym{LALR}(1) one-token lookahead it is not |
fa7e68c3 PE |
790 | possible to decide between the two forms when the identifier |
791 | @samp{a} is parsed. It is, however, desirable | |
792 | for a parser to decide this, since in the latter case | |
793 | @samp{a} must become a new identifier to represent the enumeration | |
794 | value, while in the former case @samp{a} must be evaluated with its | |
795 | current meaning, which may be a constant or even a function call. | |
796 | ||
797 | You could parse @samp{(a)} as an ``unspecified identifier in parentheses'', | |
798 | to be resolved later, but this typically requires substantial | |
799 | contortions in both semantic actions and large parts of the | |
800 | grammar, where the parentheses are nested in the recursive rules for | |
801 | expressions. | |
802 | ||
803 | You might think of using the lexer to distinguish between the two | |
804 | forms by returning different tokens for currently defined and | |
805 | undefined identifiers. But if these declarations occur in a local | |
806 | scope, and @samp{a} is defined in an outer scope, then both forms | |
807 | are possible---either locally redefining @samp{a}, or using the | |
808 | value of @samp{a} from the outer scope. So this approach cannot | |
809 | work. | |
810 | ||
e757bb10 | 811 | A simple solution to this problem is to declare the parser to |
fa7e68c3 PE |
812 | use the @acronym{GLR} algorithm. |
813 | When the @acronym{GLR} parser reaches the critical state, it | |
814 | merely splits into two branches and pursues both syntax rules | |
815 | simultaneously. Sooner or later, one of them runs into a parsing | |
816 | error. If there is a @samp{..} token before the next | |
817 | @samp{;}, the rule for enumerated types fails since it cannot | |
818 | accept @samp{..} anywhere; otherwise, the subrange type rule | |
819 | fails since it requires a @samp{..} token. So one of the branches | |
820 | fails silently, and the other one continues normally, performing | |
821 | all the intermediate actions that were postponed during the split. | |
822 | ||
823 | If the input is syntactically incorrect, both branches fail and the parser | |
824 | reports a syntax error as usual. | |
825 | ||
826 | The effect of all this is that the parser seems to ``guess'' the | |
827 | correct branch to take, or in other words, it seems to use more | |
742e4900 | 828 | lookahead than the underlying @acronym{LALR}(1) algorithm actually allows |
fa7e68c3 PE |
829 | for. In this example, @acronym{LALR}(2) would suffice, but also some cases |
830 | that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way. | |
831 | ||
832 | In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time, | |
833 | and the current Bison parser even takes exponential time and space | |
834 | for some grammars. In practice, this rarely happens, and for many | |
835 | grammars it is possible to prove that it cannot happen. | |
836 | The present example contains only one conflict between two | |
837 | rules, and the type-declaration context containing the conflict | |
838 | cannot be nested. So the number of | |
839 | branches that can exist at any time is limited by the constant 2, | |
840 | and the parsing time is still linear. | |
841 | ||
842 | Here is a Bison grammar corresponding to the example above. It | |
843 | parses a vastly simplified form of Pascal type declarations. | |
844 | ||
845 | @example | |
846 | %token TYPE DOTDOT ID | |
847 | ||
848 | @group | |
849 | %left '+' '-' | |
850 | %left '*' '/' | |
851 | @end group | |
852 | ||
853 | %% | |
854 | ||
855 | @group | |
856 | type_decl : TYPE ID '=' type ';' | |
857 | ; | |
858 | @end group | |
859 | ||
860 | @group | |
861 | type : '(' id_list ')' | |
862 | | expr DOTDOT expr | |
863 | ; | |
864 | @end group | |
865 | ||
866 | @group | |
867 | id_list : ID | |
868 | | id_list ',' ID | |
869 | ; | |
870 | @end group | |
871 | ||
872 | @group | |
873 | expr : '(' expr ')' | |
874 | | expr '+' expr | |
875 | | expr '-' expr | |
876 | | expr '*' expr | |
877 | | expr '/' expr | |
878 | | ID | |
879 | ; | |
880 | @end group | |
881 | @end example | |
882 | ||
883 | When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains | |
884 | about one reduce/reduce conflict. In the conflicting situation the | |
885 | parser chooses one of the alternatives, arbitrarily the one | |
886 | declared first. Therefore the following correct input is not | |
887 | recognized: | |
888 | ||
889 | @example | |
890 | type t = (a) .. b; | |
891 | @end example | |
892 | ||
893 | The parser can be turned into a @acronym{GLR} parser, while also telling Bison | |
894 | to be silent about the one known reduce/reduce conflict, by | |
e757bb10 | 895 | adding these two declarations to the Bison input file (before the first |
fa7e68c3 PE |
896 | @samp{%%}): |
897 | ||
898 | @example | |
899 | %glr-parser | |
900 | %expect-rr 1 | |
901 | @end example | |
902 | ||
903 | @noindent | |
904 | No change in the grammar itself is required. Now the | |
905 | parser recognizes all valid declarations, according to the | |
906 | limited syntax above, transparently. In fact, the user does not even | |
907 | notice when the parser splits. | |
908 | ||
f8e1c9e5 AD |
909 | So here we have a case where we can use the benefits of @acronym{GLR}, |
910 | almost without disadvantages. Even in simple cases like this, however, | |
911 | there are at least two potential problems to beware. First, always | |
912 | analyze the conflicts reported by Bison to make sure that @acronym{GLR} | |
913 | splitting is only done where it is intended. A @acronym{GLR} parser | |
914 | splitting inadvertently may cause problems less obvious than an | |
915 | @acronym{LALR} parser statically choosing the wrong alternative in a | |
916 | conflict. Second, consider interactions with the lexer (@pxref{Semantic | |
917 | Tokens}) with great care. Since a split parser consumes tokens without | |
918 | performing any actions during the split, the lexer cannot obtain | |
919 | information via parser actions. Some cases of lexer interactions can be | |
920 | eliminated by using @acronym{GLR} to shift the complications from the | |
921 | lexer to the parser. You must check the remaining cases for | |
922 | correctness. | |
923 | ||
924 | In our example, it would be safe for the lexer to return tokens based on | |
925 | their current meanings in some symbol table, because no new symbols are | |
926 | defined in the middle of a type declaration. Though it is possible for | |
927 | a parser to define the enumeration constants as they are parsed, before | |
928 | the type declaration is completed, it actually makes no difference since | |
929 | they cannot be used within the same enumerated type declaration. | |
fa7e68c3 PE |
930 | |
931 | @node Merging GLR Parses | |
932 | @subsection Using @acronym{GLR} to Resolve Ambiguities | |
933 | @cindex @acronym{GLR} parsing, ambiguous grammars | |
934 | @cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars | |
935 | @findex %dprec | |
936 | @findex %merge | |
937 | @cindex conflicts | |
938 | @cindex reduce/reduce conflicts | |
939 | ||
2a8d363a | 940 | Let's consider an example, vastly simplified from a C++ grammar. |
676385e2 PH |
941 | |
942 | @example | |
943 | %@{ | |
38a92d50 PE |
944 | #include <stdio.h> |
945 | #define YYSTYPE char const * | |
946 | int yylex (void); | |
947 | void yyerror (char const *); | |
676385e2 PH |
948 | %@} |
949 | ||
950 | %token TYPENAME ID | |
951 | ||
952 | %right '=' | |
953 | %left '+' | |
954 | ||
955 | %glr-parser | |
956 | ||
957 | %% | |
958 | ||
fae437e8 | 959 | prog : |
676385e2 PH |
960 | | prog stmt @{ printf ("\n"); @} |
961 | ; | |
962 | ||
963 | stmt : expr ';' %dprec 1 | |
964 | | decl %dprec 2 | |
965 | ; | |
966 | ||
2a8d363a | 967 | expr : ID @{ printf ("%s ", $$); @} |
fae437e8 | 968 | | TYPENAME '(' expr ')' |
2a8d363a AD |
969 | @{ printf ("%s <cast> ", $1); @} |
970 | | expr '+' expr @{ printf ("+ "); @} | |
971 | | expr '=' expr @{ printf ("= "); @} | |
676385e2 PH |
972 | ; |
973 | ||
fae437e8 | 974 | decl : TYPENAME declarator ';' |
2a8d363a | 975 | @{ printf ("%s <declare> ", $1); @} |
676385e2 | 976 | | TYPENAME declarator '=' expr ';' |
2a8d363a | 977 | @{ printf ("%s <init-declare> ", $1); @} |
676385e2 PH |
978 | ; |
979 | ||
2a8d363a | 980 | declarator : ID @{ printf ("\"%s\" ", $1); @} |
676385e2 PH |
981 | | '(' declarator ')' |
982 | ; | |
983 | @end example | |
984 | ||
985 | @noindent | |
986 | This models a problematic part of the C++ grammar---the ambiguity between | |
987 | certain declarations and statements. For example, | |
988 | ||
989 | @example | |
990 | T (x) = y+z; | |
991 | @end example | |
992 | ||
993 | @noindent | |
994 | parses as either an @code{expr} or a @code{stmt} | |
c827f760 PE |
995 | (assuming that @samp{T} is recognized as a @code{TYPENAME} and |
996 | @samp{x} as an @code{ID}). | |
676385e2 | 997 | Bison detects this as a reduce/reduce conflict between the rules |
fae437e8 | 998 | @code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the |
e757bb10 AD |
999 | time it encounters @code{x} in the example above. Since this is a |
1000 | @acronym{GLR} parser, it therefore splits the problem into two parses, one for | |
fa7e68c3 PE |
1001 | each choice of resolving the reduce/reduce conflict. |
1002 | Unlike the example from the previous section (@pxref{Simple GLR Parsers}), | |
1003 | however, neither of these parses ``dies,'' because the grammar as it stands is | |
e757bb10 AD |
1004 | ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and |
1005 | the other reduces @code{stmt : decl}, after which both parsers are in an | |
1006 | identical state: they've seen @samp{prog stmt} and have the same unprocessed | |
1007 | input remaining. We say that these parses have @dfn{merged.} | |
fa7e68c3 PE |
1008 | |
1009 | At this point, the @acronym{GLR} parser requires a specification in the | |
1010 | grammar of how to choose between the competing parses. | |
1011 | In the example above, the two @code{%dprec} | |
e757bb10 | 1012 | declarations specify that Bison is to give precedence |
fa7e68c3 | 1013 | to the parse that interprets the example as a |
676385e2 PH |
1014 | @code{decl}, which implies that @code{x} is a declarator. |
1015 | The parser therefore prints | |
1016 | ||
1017 | @example | |
fae437e8 | 1018 | "x" y z + T <init-declare> |
676385e2 PH |
1019 | @end example |
1020 | ||
fa7e68c3 PE |
1021 | The @code{%dprec} declarations only come into play when more than one |
1022 | parse survives. Consider a different input string for this parser: | |
676385e2 PH |
1023 | |
1024 | @example | |
1025 | T (x) + y; | |
1026 | @end example | |
1027 | ||
1028 | @noindent | |
e757bb10 | 1029 | This is another example of using @acronym{GLR} to parse an unambiguous |
fa7e68c3 | 1030 | construct, as shown in the previous section (@pxref{Simple GLR Parsers}). |
676385e2 PH |
1031 | Here, there is no ambiguity (this cannot be parsed as a declaration). |
1032 | However, at the time the Bison parser encounters @code{x}, it does not | |
1033 | have enough information to resolve the reduce/reduce conflict (again, | |
1034 | between @code{x} as an @code{expr} or a @code{declarator}). In this | |
fa7e68c3 | 1035 | case, no precedence declaration is used. Again, the parser splits |
676385e2 PH |
1036 | into two, one assuming that @code{x} is an @code{expr}, and the other |
1037 | assuming @code{x} is a @code{declarator}. The second of these parsers | |
1038 | then vanishes when it sees @code{+}, and the parser prints | |
1039 | ||
1040 | @example | |
fae437e8 | 1041 | x T <cast> y + |
676385e2 PH |
1042 | @end example |
1043 | ||
1044 | Suppose that instead of resolving the ambiguity, you wanted to see all | |
fa7e68c3 | 1045 | the possibilities. For this purpose, you must merge the semantic |
676385e2 PH |
1046 | actions of the two possible parsers, rather than choosing one over the |
1047 | other. To do so, you could change the declaration of @code{stmt} as | |
1048 | follows: | |
1049 | ||
1050 | @example | |
1051 | stmt : expr ';' %merge <stmtMerge> | |
1052 | | decl %merge <stmtMerge> | |
1053 | ; | |
1054 | @end example | |
1055 | ||
1056 | @noindent | |
676385e2 PH |
1057 | and define the @code{stmtMerge} function as: |
1058 | ||
1059 | @example | |
38a92d50 PE |
1060 | static YYSTYPE |
1061 | stmtMerge (YYSTYPE x0, YYSTYPE x1) | |
676385e2 PH |
1062 | @{ |
1063 | printf ("<OR> "); | |
1064 | return ""; | |
1065 | @} | |
1066 | @end example | |
1067 | ||
1068 | @noindent | |
1069 | with an accompanying forward declaration | |
1070 | in the C declarations at the beginning of the file: | |
1071 | ||
1072 | @example | |
1073 | %@{ | |
38a92d50 | 1074 | #define YYSTYPE char const * |
676385e2 PH |
1075 | static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1); |
1076 | %@} | |
1077 | @end example | |
1078 | ||
1079 | @noindent | |
fa7e68c3 PE |
1080 | With these declarations, the resulting parser parses the first example |
1081 | as both an @code{expr} and a @code{decl}, and prints | |
676385e2 PH |
1082 | |
1083 | @example | |
fae437e8 | 1084 | "x" y z + T <init-declare> x T <cast> y z + = <OR> |
676385e2 PH |
1085 | @end example |
1086 | ||
fa7e68c3 | 1087 | Bison requires that all of the |
e757bb10 | 1088 | productions that participate in any particular merge have identical |
fa7e68c3 PE |
1089 | @samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable, |
1090 | and the parser will report an error during any parse that results in | |
1091 | the offending merge. | |
9501dc6e | 1092 | |
32c29292 JD |
1093 | @node GLR Semantic Actions |
1094 | @subsection GLR Semantic Actions | |
1095 | ||
1096 | @cindex deferred semantic actions | |
1097 | By definition, a deferred semantic action is not performed at the same time as | |
1098 | the associated reduction. | |
1099 | This raises caveats for several Bison features you might use in a semantic | |
1100 | action in a @acronym{GLR} parser. | |
1101 | ||
1102 | @vindex yychar | |
1103 | @cindex @acronym{GLR} parsers and @code{yychar} | |
1104 | @vindex yylval | |
1105 | @cindex @acronym{GLR} parsers and @code{yylval} | |
1106 | @vindex yylloc | |
1107 | @cindex @acronym{GLR} parsers and @code{yylloc} | |
1108 | In any semantic action, you can examine @code{yychar} to determine the type of | |
742e4900 | 1109 | the lookahead token present at the time of the associated reduction. |
32c29292 JD |
1110 | After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF}, |
1111 | you can then examine @code{yylval} and @code{yylloc} to determine the | |
742e4900 | 1112 | lookahead token's semantic value and location, if any. |
32c29292 JD |
1113 | In a nondeferred semantic action, you can also modify any of these variables to |
1114 | influence syntax analysis. | |
742e4900 | 1115 | @xref{Lookahead, ,Lookahead Tokens}. |
32c29292 JD |
1116 | |
1117 | @findex yyclearin | |
1118 | @cindex @acronym{GLR} parsers and @code{yyclearin} | |
1119 | In a deferred semantic action, it's too late to influence syntax analysis. | |
1120 | In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to | |
1121 | shallow copies of the values they had at the time of the associated reduction. | |
1122 | For this reason alone, modifying them is dangerous. | |
1123 | Moreover, the result of modifying them is undefined and subject to change with | |
1124 | future versions of Bison. | |
1125 | For example, if a semantic action might be deferred, you should never write it | |
1126 | to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free | |
1127 | memory referenced by @code{yylval}. | |
1128 | ||
1129 | @findex YYERROR | |
1130 | @cindex @acronym{GLR} parsers and @code{YYERROR} | |
1131 | Another Bison feature requiring special consideration is @code{YYERROR} | |
8710fc41 | 1132 | (@pxref{Action Features}), which you can invoke in a semantic action to |
32c29292 JD |
1133 | initiate error recovery. |
1134 | During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is | |
1135 | the same as its effect in an @acronym{LALR}(1) parser. | |
1136 | In a deferred semantic action, its effect is undefined. | |
1137 | @c The effect is probably a syntax error at the split point. | |
1138 | ||
8710fc41 JD |
1139 | Also, see @ref{Location Default Action, ,Default Action for Locations}, which |
1140 | describes a special usage of @code{YYLLOC_DEFAULT} in @acronym{GLR} parsers. | |
1141 | ||
fa7e68c3 PE |
1142 | @node Compiler Requirements |
1143 | @subsection Considerations when Compiling @acronym{GLR} Parsers | |
1144 | @cindex @code{inline} | |
9501dc6e | 1145 | @cindex @acronym{GLR} parsers and @code{inline} |
fa7e68c3 | 1146 | |
38a92d50 PE |
1147 | The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or |
1148 | later. In addition, they use the @code{inline} keyword, which is not | |
1149 | C89, but is C99 and is a common extension in pre-C99 compilers. It is | |
1150 | up to the user of these parsers to handle | |
9501dc6e AD |
1151 | portability issues. For instance, if using Autoconf and the Autoconf |
1152 | macro @code{AC_C_INLINE}, a mere | |
1153 | ||
1154 | @example | |
1155 | %@{ | |
38a92d50 | 1156 | #include <config.h> |
9501dc6e AD |
1157 | %@} |
1158 | @end example | |
1159 | ||
1160 | @noindent | |
1161 | will suffice. Otherwise, we suggest | |
1162 | ||
1163 | @example | |
1164 | %@{ | |
38a92d50 PE |
1165 | #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline |
1166 | #define inline | |
1167 | #endif | |
9501dc6e AD |
1168 | %@} |
1169 | @end example | |
676385e2 | 1170 | |
342b8b6e | 1171 | @node Locations Overview |
847bf1f5 AD |
1172 | @section Locations |
1173 | @cindex location | |
95923bd6 AD |
1174 | @cindex textual location |
1175 | @cindex location, textual | |
847bf1f5 AD |
1176 | |
1177 | Many applications, like interpreters or compilers, have to produce verbose | |
72d2299c | 1178 | and useful error messages. To achieve this, one must be able to keep track of |
95923bd6 | 1179 | the @dfn{textual location}, or @dfn{location}, of each syntactic construct. |
847bf1f5 AD |
1180 | Bison provides a mechanism for handling these locations. |
1181 | ||
72d2299c | 1182 | Each token has a semantic value. In a similar fashion, each token has an |
847bf1f5 | 1183 | associated location, but the type of locations is the same for all tokens and |
72d2299c | 1184 | groupings. Moreover, the output parser is equipped with a default data |
847bf1f5 AD |
1185 | structure for storing locations (@pxref{Locations}, for more details). |
1186 | ||
1187 | Like semantic values, locations can be reached in actions using a dedicated | |
72d2299c | 1188 | set of constructs. In the example above, the location of the whole grouping |
847bf1f5 AD |
1189 | is @code{@@$}, while the locations of the subexpressions are @code{@@1} and |
1190 | @code{@@3}. | |
1191 | ||
1192 | When a rule is matched, a default action is used to compute the semantic value | |
72d2299c PE |
1193 | of its left hand side (@pxref{Actions}). In the same way, another default |
1194 | action is used for locations. However, the action for locations is general | |
847bf1f5 | 1195 | enough for most cases, meaning there is usually no need to describe for each |
72d2299c | 1196 | rule how @code{@@$} should be formed. When building a new location for a given |
847bf1f5 AD |
1197 | grouping, the default behavior of the output parser is to take the beginning |
1198 | of the first symbol, and the end of the last symbol. | |
1199 | ||
342b8b6e | 1200 | @node Bison Parser |
bfa74976 RS |
1201 | @section Bison Output: the Parser File |
1202 | @cindex Bison parser | |
1203 | @cindex Bison utility | |
1204 | @cindex lexical analyzer, purpose | |
1205 | @cindex parser | |
1206 | ||
1207 | When you run Bison, you give it a Bison grammar file as input. The output | |
1208 | is a C source file that parses the language described by the grammar. | |
1209 | This file is called a @dfn{Bison parser}. Keep in mind that the Bison | |
1210 | utility and the Bison parser are two distinct programs: the Bison utility | |
1211 | is a program whose output is the Bison parser that becomes part of your | |
1212 | program. | |
1213 | ||
1214 | The job of the Bison parser is to group tokens into groupings according to | |
1215 | the grammar rules---for example, to build identifiers and operators into | |
1216 | expressions. As it does this, it runs the actions for the grammar rules it | |
1217 | uses. | |
1218 | ||
704a47c4 AD |
1219 | The tokens come from a function called the @dfn{lexical analyzer} that |
1220 | you must supply in some fashion (such as by writing it in C). The Bison | |
1221 | parser calls the lexical analyzer each time it wants a new token. It | |
1222 | doesn't know what is ``inside'' the tokens (though their semantic values | |
1223 | may reflect this). Typically the lexical analyzer makes the tokens by | |
1224 | parsing characters of text, but Bison does not depend on this. | |
1225 | @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}. | |
bfa74976 RS |
1226 | |
1227 | The Bison parser file is C code which defines a function named | |
1228 | @code{yyparse} which implements that grammar. This function does not make | |
1229 | a complete C program: you must supply some additional functions. One is | |
1230 | the lexical analyzer. Another is an error-reporting function which the | |
1231 | parser calls to report an error. In addition, a complete C program must | |
1232 | start with a function called @code{main}; you have to provide this, and | |
1233 | arrange for it to call @code{yyparse} or the parser will never run. | |
1234 | @xref{Interface, ,Parser C-Language Interface}. | |
1235 | ||
f7ab6a50 | 1236 | Aside from the token type names and the symbols in the actions you |
7093d0f5 | 1237 | write, all symbols defined in the Bison parser file itself |
bfa74976 RS |
1238 | begin with @samp{yy} or @samp{YY}. This includes interface functions |
1239 | such as the lexical analyzer function @code{yylex}, the error reporting | |
1240 | function @code{yyerror} and the parser function @code{yyparse} itself. | |
1241 | This also includes numerous identifiers used for internal purposes. | |
1242 | Therefore, you should avoid using C identifiers starting with @samp{yy} | |
1243 | or @samp{YY} in the Bison grammar file except for the ones defined in | |
55289366 PE |
1244 | this manual. Also, you should avoid using the C identifiers |
1245 | @samp{malloc} and @samp{free} for anything other than their usual | |
1246 | meanings. | |
bfa74976 | 1247 | |
7093d0f5 AD |
1248 | In some cases the Bison parser file includes system headers, and in |
1249 | those cases your code should respect the identifiers reserved by those | |
55289366 | 1250 | headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>}, |
7093d0f5 | 1251 | @code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to |
30757c8c PE |
1252 | declare memory allocators and related types. @code{<libintl.h>} is |
1253 | included if message translation is in use | |
1254 | (@pxref{Internationalization}). Other system headers may | |
ec3bc396 AD |
1255 | be included if you define @code{YYDEBUG} to a nonzero value |
1256 | (@pxref{Tracing, ,Tracing Your Parser}). | |
7093d0f5 | 1257 | |
342b8b6e | 1258 | @node Stages |
bfa74976 RS |
1259 | @section Stages in Using Bison |
1260 | @cindex stages in using Bison | |
1261 | @cindex using Bison | |
1262 | ||
1263 | The actual language-design process using Bison, from grammar specification | |
1264 | to a working compiler or interpreter, has these parts: | |
1265 | ||
1266 | @enumerate | |
1267 | @item | |
1268 | Formally specify the grammar in a form recognized by Bison | |
704a47c4 AD |
1269 | (@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule |
1270 | in the language, describe the action that is to be taken when an | |
1271 | instance of that rule is recognized. The action is described by a | |
1272 | sequence of C statements. | |
bfa74976 RS |
1273 | |
1274 | @item | |
704a47c4 AD |
1275 | Write a lexical analyzer to process input and pass tokens to the parser. |
1276 | The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The | |
1277 | Lexical Analyzer Function @code{yylex}}). It could also be produced | |
1278 | using Lex, but the use of Lex is not discussed in this manual. | |
bfa74976 RS |
1279 | |
1280 | @item | |
1281 | Write a controlling function that calls the Bison-produced parser. | |
1282 | ||
1283 | @item | |
1284 | Write error-reporting routines. | |
1285 | @end enumerate | |
1286 | ||
1287 | To turn this source code as written into a runnable program, you | |
1288 | must follow these steps: | |
1289 | ||
1290 | @enumerate | |
1291 | @item | |
1292 | Run Bison on the grammar to produce the parser. | |
1293 | ||
1294 | @item | |
1295 | Compile the code output by Bison, as well as any other source files. | |
1296 | ||
1297 | @item | |
1298 | Link the object files to produce the finished product. | |
1299 | @end enumerate | |
1300 | ||
342b8b6e | 1301 | @node Grammar Layout |
bfa74976 RS |
1302 | @section The Overall Layout of a Bison Grammar |
1303 | @cindex grammar file | |
1304 | @cindex file format | |
1305 | @cindex format of grammar file | |
1306 | @cindex layout of Bison grammar | |
1307 | ||
1308 | The input file for the Bison utility is a @dfn{Bison grammar file}. The | |
1309 | general form of a Bison grammar file is as follows: | |
1310 | ||
1311 | @example | |
1312 | %@{ | |
08e49d20 | 1313 | @var{Prologue} |
bfa74976 RS |
1314 | %@} |
1315 | ||
1316 | @var{Bison declarations} | |
1317 | ||
1318 | %% | |
1319 | @var{Grammar rules} | |
1320 | %% | |
08e49d20 | 1321 | @var{Epilogue} |
bfa74976 RS |
1322 | @end example |
1323 | ||
1324 | @noindent | |
1325 | The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears | |
1326 | in every Bison grammar file to separate the sections. | |
1327 | ||
72d2299c | 1328 | The prologue may define types and variables used in the actions. You can |
342b8b6e | 1329 | also use preprocessor commands to define macros used there, and use |
bfa74976 | 1330 | @code{#include} to include header files that do any of these things. |
38a92d50 PE |
1331 | You need to declare the lexical analyzer @code{yylex} and the error |
1332 | printer @code{yyerror} here, along with any other global identifiers | |
1333 | used by the actions in the grammar rules. | |
bfa74976 RS |
1334 | |
1335 | The Bison declarations declare the names of the terminal and nonterminal | |
1336 | symbols, and may also describe operator precedence and the data types of | |
1337 | semantic values of various symbols. | |
1338 | ||
1339 | The grammar rules define how to construct each nonterminal symbol from its | |
1340 | parts. | |
1341 | ||
38a92d50 PE |
1342 | The epilogue can contain any code you want to use. Often the |
1343 | definitions of functions declared in the prologue go here. In a | |
1344 | simple program, all the rest of the program can go here. | |
bfa74976 | 1345 | |
342b8b6e | 1346 | @node Examples |
bfa74976 RS |
1347 | @chapter Examples |
1348 | @cindex simple examples | |
1349 | @cindex examples, simple | |
1350 | ||
1351 | Now we show and explain three sample programs written using Bison: a | |
1352 | reverse polish notation calculator, an algebraic (infix) notation | |
1353 | calculator, and a multi-function calculator. All three have been tested | |
1354 | under BSD Unix 4.3; each produces a usable, though limited, interactive | |
1355 | desk-top calculator. | |
1356 | ||
1357 | These examples are simple, but Bison grammars for real programming | |
aa08666d AD |
1358 | languages are written the same way. You can copy these examples into a |
1359 | source file to try them. | |
bfa74976 RS |
1360 | |
1361 | @menu | |
1362 | * RPN Calc:: Reverse polish notation calculator; | |
1363 | a first example with no operator precedence. | |
1364 | * Infix Calc:: Infix (algebraic) notation calculator. | |
1365 | Operator precedence is introduced. | |
1366 | * Simple Error Recovery:: Continuing after syntax errors. | |
342b8b6e | 1367 | * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$. |
bfa74976 RS |
1368 | * Multi-function Calc:: Calculator with memory and trig functions. |
1369 | It uses multiple data-types for semantic values. | |
1370 | * Exercises:: Ideas for improving the multi-function calculator. | |
1371 | @end menu | |
1372 | ||
342b8b6e | 1373 | @node RPN Calc |
bfa74976 RS |
1374 | @section Reverse Polish Notation Calculator |
1375 | @cindex reverse polish notation | |
1376 | @cindex polish notation calculator | |
1377 | @cindex @code{rpcalc} | |
1378 | @cindex calculator, simple | |
1379 | ||
1380 | The first example is that of a simple double-precision @dfn{reverse polish | |
1381 | notation} calculator (a calculator using postfix operators). This example | |
1382 | provides a good starting point, since operator precedence is not an issue. | |
1383 | The second example will illustrate how operator precedence is handled. | |
1384 | ||
1385 | The source code for this calculator is named @file{rpcalc.y}. The | |
1386 | @samp{.y} extension is a convention used for Bison input files. | |
1387 | ||
1388 | @menu | |
75f5aaea | 1389 | * Decls: Rpcalc Decls. Prologue (declarations) for rpcalc. |
bfa74976 RS |
1390 | * Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation. |
1391 | * Lexer: Rpcalc Lexer. The lexical analyzer. | |
1392 | * Main: Rpcalc Main. The controlling function. | |
1393 | * Error: Rpcalc Error. The error reporting function. | |
1394 | * Gen: Rpcalc Gen. Running Bison on the grammar file. | |
1395 | * Comp: Rpcalc Compile. Run the C compiler on the output code. | |
1396 | @end menu | |
1397 | ||
342b8b6e | 1398 | @node Rpcalc Decls |
bfa74976 RS |
1399 | @subsection Declarations for @code{rpcalc} |
1400 | ||
1401 | Here are the C and Bison declarations for the reverse polish notation | |
1402 | calculator. As in C, comments are placed between @samp{/*@dots{}*/}. | |
1403 | ||
1404 | @example | |
72d2299c | 1405 | /* Reverse polish notation calculator. */ |
bfa74976 RS |
1406 | |
1407 | %@{ | |
38a92d50 PE |
1408 | #define YYSTYPE double |
1409 | #include <math.h> | |
1410 | int yylex (void); | |
1411 | void yyerror (char const *); | |
bfa74976 RS |
1412 | %@} |
1413 | ||
1414 | %token NUM | |
1415 | ||
72d2299c | 1416 | %% /* Grammar rules and actions follow. */ |
bfa74976 RS |
1417 | @end example |
1418 | ||
75f5aaea | 1419 | The declarations section (@pxref{Prologue, , The prologue}) contains two |
38a92d50 | 1420 | preprocessor directives and two forward declarations. |
bfa74976 RS |
1421 | |
1422 | The @code{#define} directive defines the macro @code{YYSTYPE}, thus | |
1964ad8c AD |
1423 | specifying the C data type for semantic values of both tokens and |
1424 | groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The | |
1425 | Bison parser will use whatever type @code{YYSTYPE} is defined as; if you | |
1426 | don't define it, @code{int} is the default. Because we specify | |
1427 | @code{double}, each token and each expression has an associated value, | |
1428 | which is a floating point number. | |
bfa74976 RS |
1429 | |
1430 | The @code{#include} directive is used to declare the exponentiation | |
1431 | function @code{pow}. | |
1432 | ||
38a92d50 PE |
1433 | The forward declarations for @code{yylex} and @code{yyerror} are |
1434 | needed because the C language requires that functions be declared | |
1435 | before they are used. These functions will be defined in the | |
1436 | epilogue, but the parser calls them so they must be declared in the | |
1437 | prologue. | |
1438 | ||
704a47c4 AD |
1439 | The second section, Bison declarations, provides information to Bison |
1440 | about the token types (@pxref{Bison Declarations, ,The Bison | |
1441 | Declarations Section}). Each terminal symbol that is not a | |
1442 | single-character literal must be declared here. (Single-character | |
bfa74976 RS |
1443 | literals normally don't need to be declared.) In this example, all the |
1444 | arithmetic operators are designated by single-character literals, so the | |
1445 | only terminal symbol that needs to be declared is @code{NUM}, the token | |
1446 | type for numeric constants. | |
1447 | ||
342b8b6e | 1448 | @node Rpcalc Rules |
bfa74976 RS |
1449 | @subsection Grammar Rules for @code{rpcalc} |
1450 | ||
1451 | Here are the grammar rules for the reverse polish notation calculator. | |
1452 | ||
1453 | @example | |
1454 | input: /* empty */ | |
1455 | | input line | |
1456 | ; | |
1457 | ||
1458 | line: '\n' | |
18b519c0 | 1459 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} |
bfa74976 RS |
1460 | ; |
1461 | ||
18b519c0 AD |
1462 | exp: NUM @{ $$ = $1; @} |
1463 | | exp exp '+' @{ $$ = $1 + $2; @} | |
1464 | | exp exp '-' @{ $$ = $1 - $2; @} | |
1465 | | exp exp '*' @{ $$ = $1 * $2; @} | |
1466 | | exp exp '/' @{ $$ = $1 / $2; @} | |
1467 | /* Exponentiation */ | |
1468 | | exp exp '^' @{ $$ = pow ($1, $2); @} | |
1469 | /* Unary minus */ | |
1470 | | exp 'n' @{ $$ = -$1; @} | |
bfa74976 RS |
1471 | ; |
1472 | %% | |
1473 | @end example | |
1474 | ||
1475 | The groupings of the rpcalc ``language'' defined here are the expression | |
1476 | (given the name @code{exp}), the line of input (@code{line}), and the | |
1477 | complete input transcript (@code{input}). Each of these nonterminal | |
8c5b881d | 1478 | symbols has several alternate rules, joined by the vertical bar @samp{|} |
bfa74976 RS |
1479 | which is read as ``or''. The following sections explain what these rules |
1480 | mean. | |
1481 | ||
1482 | The semantics of the language is determined by the actions taken when a | |
1483 | grouping is recognized. The actions are the C code that appears inside | |
1484 | braces. @xref{Actions}. | |
1485 | ||
1486 | You must specify these actions in C, but Bison provides the means for | |
1487 | passing semantic values between the rules. In each action, the | |
1488 | pseudo-variable @code{$$} stands for the semantic value for the grouping | |
1489 | that the rule is going to construct. Assigning a value to @code{$$} is the | |
1490 | main job of most actions. The semantic values of the components of the | |
1491 | rule are referred to as @code{$1}, @code{$2}, and so on. | |
1492 | ||
1493 | @menu | |
13863333 AD |
1494 | * Rpcalc Input:: |
1495 | * Rpcalc Line:: | |
1496 | * Rpcalc Expr:: | |
bfa74976 RS |
1497 | @end menu |
1498 | ||
342b8b6e | 1499 | @node Rpcalc Input |
bfa74976 RS |
1500 | @subsubsection Explanation of @code{input} |
1501 | ||
1502 | Consider the definition of @code{input}: | |
1503 | ||
1504 | @example | |
1505 | input: /* empty */ | |
1506 | | input line | |
1507 | ; | |
1508 | @end example | |
1509 | ||
1510 | This definition reads as follows: ``A complete input is either an empty | |
1511 | string, or a complete input followed by an input line''. Notice that | |
1512 | ``complete input'' is defined in terms of itself. This definition is said | |
1513 | to be @dfn{left recursive} since @code{input} appears always as the | |
1514 | leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}. | |
1515 | ||
1516 | The first alternative is empty because there are no symbols between the | |
1517 | colon and the first @samp{|}; this means that @code{input} can match an | |
1518 | empty string of input (no tokens). We write the rules this way because it | |
1519 | is legitimate to type @kbd{Ctrl-d} right after you start the calculator. | |
1520 | It's conventional to put an empty alternative first and write the comment | |
1521 | @samp{/* empty */} in it. | |
1522 | ||
1523 | The second alternate rule (@code{input line}) handles all nontrivial input. | |
1524 | It means, ``After reading any number of lines, read one more line if | |
1525 | possible.'' The left recursion makes this rule into a loop. Since the | |
1526 | first alternative matches empty input, the loop can be executed zero or | |
1527 | more times. | |
1528 | ||
1529 | The parser function @code{yyparse} continues to process input until a | |
1530 | grammatical error is seen or the lexical analyzer says there are no more | |
72d2299c | 1531 | input tokens; we will arrange for the latter to happen at end-of-input. |
bfa74976 | 1532 | |
342b8b6e | 1533 | @node Rpcalc Line |
bfa74976 RS |
1534 | @subsubsection Explanation of @code{line} |
1535 | ||
1536 | Now consider the definition of @code{line}: | |
1537 | ||
1538 | @example | |
1539 | line: '\n' | |
1540 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
1541 | ; | |
1542 | @end example | |
1543 | ||
1544 | The first alternative is a token which is a newline character; this means | |
1545 | that rpcalc accepts a blank line (and ignores it, since there is no | |
1546 | action). The second alternative is an expression followed by a newline. | |
1547 | This is the alternative that makes rpcalc useful. The semantic value of | |
1548 | the @code{exp} grouping is the value of @code{$1} because the @code{exp} in | |
1549 | question is the first symbol in the alternative. The action prints this | |
1550 | value, which is the result of the computation the user asked for. | |
1551 | ||
1552 | This action is unusual because it does not assign a value to @code{$$}. As | |
1553 | a consequence, the semantic value associated with the @code{line} is | |
1554 | uninitialized (its value will be unpredictable). This would be a bug if | |
1555 | that value were ever used, but we don't use it: once rpcalc has printed the | |
1556 | value of the user's input line, that value is no longer needed. | |
1557 | ||
342b8b6e | 1558 | @node Rpcalc Expr |
bfa74976 RS |
1559 | @subsubsection Explanation of @code{expr} |
1560 | ||
1561 | The @code{exp} grouping has several rules, one for each kind of expression. | |
1562 | The first rule handles the simplest expressions: those that are just numbers. | |
1563 | The second handles an addition-expression, which looks like two expressions | |
1564 | followed by a plus-sign. The third handles subtraction, and so on. | |
1565 | ||
1566 | @example | |
1567 | exp: NUM | |
1568 | | exp exp '+' @{ $$ = $1 + $2; @} | |
1569 | | exp exp '-' @{ $$ = $1 - $2; @} | |
1570 | @dots{} | |
1571 | ; | |
1572 | @end example | |
1573 | ||
1574 | We have used @samp{|} to join all the rules for @code{exp}, but we could | |
1575 | equally well have written them separately: | |
1576 | ||
1577 | @example | |
1578 | exp: NUM ; | |
1579 | exp: exp exp '+' @{ $$ = $1 + $2; @} ; | |
1580 | exp: exp exp '-' @{ $$ = $1 - $2; @} ; | |
1581 | @dots{} | |
1582 | @end example | |
1583 | ||
1584 | Most of the rules have actions that compute the value of the expression in | |
1585 | terms of the value of its parts. For example, in the rule for addition, | |
1586 | @code{$1} refers to the first component @code{exp} and @code{$2} refers to | |
1587 | the second one. The third component, @code{'+'}, has no meaningful | |
1588 | associated semantic value, but if it had one you could refer to it as | |
1589 | @code{$3}. When @code{yyparse} recognizes a sum expression using this | |
1590 | rule, the sum of the two subexpressions' values is produced as the value of | |
1591 | the entire expression. @xref{Actions}. | |
1592 | ||
1593 | You don't have to give an action for every rule. When a rule has no | |
1594 | action, Bison by default copies the value of @code{$1} into @code{$$}. | |
1595 | This is what happens in the first rule (the one that uses @code{NUM}). | |
1596 | ||
1597 | The formatting shown here is the recommended convention, but Bison does | |
72d2299c | 1598 | not require it. You can add or change white space as much as you wish. |
bfa74976 RS |
1599 | For example, this: |
1600 | ||
1601 | @example | |
99a9344e | 1602 | exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ; |
bfa74976 RS |
1603 | @end example |
1604 | ||
1605 | @noindent | |
1606 | means the same thing as this: | |
1607 | ||
1608 | @example | |
1609 | exp: NUM | |
1610 | | exp exp '+' @{ $$ = $1 + $2; @} | |
1611 | | @dots{} | |
99a9344e | 1612 | ; |
bfa74976 RS |
1613 | @end example |
1614 | ||
1615 | @noindent | |
1616 | The latter, however, is much more readable. | |
1617 | ||
342b8b6e | 1618 | @node Rpcalc Lexer |
bfa74976 RS |
1619 | @subsection The @code{rpcalc} Lexical Analyzer |
1620 | @cindex writing a lexical analyzer | |
1621 | @cindex lexical analyzer, writing | |
1622 | ||
704a47c4 AD |
1623 | The lexical analyzer's job is low-level parsing: converting characters |
1624 | or sequences of characters into tokens. The Bison parser gets its | |
1625 | tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical | |
1626 | Analyzer Function @code{yylex}}. | |
bfa74976 | 1627 | |
c827f760 PE |
1628 | Only a simple lexical analyzer is needed for the @acronym{RPN} |
1629 | calculator. This | |
bfa74976 RS |
1630 | lexical analyzer skips blanks and tabs, then reads in numbers as |
1631 | @code{double} and returns them as @code{NUM} tokens. Any other character | |
1632 | that isn't part of a number is a separate token. Note that the token-code | |
1633 | for such a single-character token is the character itself. | |
1634 | ||
1635 | The return value of the lexical analyzer function is a numeric code which | |
1636 | represents a token type. The same text used in Bison rules to stand for | |
1637 | this token type is also a C expression for the numeric code for the type. | |
1638 | This works in two ways. If the token type is a character literal, then its | |
e966383b | 1639 | numeric code is that of the character; you can use the same |
bfa74976 RS |
1640 | character literal in the lexical analyzer to express the number. If the |
1641 | token type is an identifier, that identifier is defined by Bison as a C | |
1642 | macro whose definition is the appropriate number. In this example, | |
1643 | therefore, @code{NUM} becomes a macro for @code{yylex} to use. | |
1644 | ||
1964ad8c AD |
1645 | The semantic value of the token (if it has one) is stored into the |
1646 | global variable @code{yylval}, which is where the Bison parser will look | |
1647 | for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was | |
1648 | defined at the beginning of the grammar; @pxref{Rpcalc Decls, | |
1649 | ,Declarations for @code{rpcalc}}.) | |
bfa74976 | 1650 | |
72d2299c PE |
1651 | A token type code of zero is returned if the end-of-input is encountered. |
1652 | (Bison recognizes any nonpositive value as indicating end-of-input.) | |
bfa74976 RS |
1653 | |
1654 | Here is the code for the lexical analyzer: | |
1655 | ||
1656 | @example | |
1657 | @group | |
72d2299c | 1658 | /* The lexical analyzer returns a double floating point |
e966383b | 1659 | number on the stack and the token NUM, or the numeric code |
72d2299c PE |
1660 | of the character read if not a number. It skips all blanks |
1661 | and tabs, and returns 0 for end-of-input. */ | |
bfa74976 RS |
1662 | |
1663 | #include <ctype.h> | |
1664 | @end group | |
1665 | ||
1666 | @group | |
13863333 AD |
1667 | int |
1668 | yylex (void) | |
bfa74976 RS |
1669 | @{ |
1670 | int c; | |
1671 | ||
72d2299c | 1672 | /* Skip white space. */ |
13863333 | 1673 | while ((c = getchar ()) == ' ' || c == '\t') |
bfa74976 RS |
1674 | ; |
1675 | @end group | |
1676 | @group | |
72d2299c | 1677 | /* Process numbers. */ |
13863333 | 1678 | if (c == '.' || isdigit (c)) |
bfa74976 RS |
1679 | @{ |
1680 | ungetc (c, stdin); | |
1681 | scanf ("%lf", &yylval); | |
1682 | return NUM; | |
1683 | @} | |
1684 | @end group | |
1685 | @group | |
72d2299c | 1686 | /* Return end-of-input. */ |
13863333 | 1687 | if (c == EOF) |
bfa74976 | 1688 | return 0; |
72d2299c | 1689 | /* Return a single char. */ |
13863333 | 1690 | return c; |
bfa74976 RS |
1691 | @} |
1692 | @end group | |
1693 | @end example | |
1694 | ||
342b8b6e | 1695 | @node Rpcalc Main |
bfa74976 RS |
1696 | @subsection The Controlling Function |
1697 | @cindex controlling function | |
1698 | @cindex main function in simple example | |
1699 | ||
1700 | In keeping with the spirit of this example, the controlling function is | |
1701 | kept to the bare minimum. The only requirement is that it call | |
1702 | @code{yyparse} to start the process of parsing. | |
1703 | ||
1704 | @example | |
1705 | @group | |
13863333 AD |
1706 | int |
1707 | main (void) | |
bfa74976 | 1708 | @{ |
13863333 | 1709 | return yyparse (); |
bfa74976 RS |
1710 | @} |
1711 | @end group | |
1712 | @end example | |
1713 | ||
342b8b6e | 1714 | @node Rpcalc Error |
bfa74976 RS |
1715 | @subsection The Error Reporting Routine |
1716 | @cindex error reporting routine | |
1717 | ||
1718 | When @code{yyparse} detects a syntax error, it calls the error reporting | |
13863333 | 1719 | function @code{yyerror} to print an error message (usually but not |
6e649e65 | 1720 | always @code{"syntax error"}). It is up to the programmer to supply |
13863333 AD |
1721 | @code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so |
1722 | here is the definition we will use: | |
bfa74976 RS |
1723 | |
1724 | @example | |
1725 | @group | |
1726 | #include <stdio.h> | |
1727 | ||
38a92d50 | 1728 | /* Called by yyparse on error. */ |
13863333 | 1729 | void |
38a92d50 | 1730 | yyerror (char const *s) |
bfa74976 | 1731 | @{ |
4e03e201 | 1732 | fprintf (stderr, "%s\n", s); |
bfa74976 RS |
1733 | @} |
1734 | @end group | |
1735 | @end example | |
1736 | ||
1737 | After @code{yyerror} returns, the Bison parser may recover from the error | |
1738 | and continue parsing if the grammar contains a suitable error rule | |
1739 | (@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We | |
1740 | have not written any error rules in this example, so any invalid input will | |
1741 | cause the calculator program to exit. This is not clean behavior for a | |
9ecbd125 | 1742 | real calculator, but it is adequate for the first example. |
bfa74976 | 1743 | |
342b8b6e | 1744 | @node Rpcalc Gen |
bfa74976 RS |
1745 | @subsection Running Bison to Make the Parser |
1746 | @cindex running Bison (introduction) | |
1747 | ||
ceed8467 AD |
1748 | Before running Bison to produce a parser, we need to decide how to |
1749 | arrange all the source code in one or more source files. For such a | |
1750 | simple example, the easiest thing is to put everything in one file. The | |
1751 | definitions of @code{yylex}, @code{yyerror} and @code{main} go at the | |
342b8b6e | 1752 | end, in the epilogue of the file |
75f5aaea | 1753 | (@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}). |
bfa74976 RS |
1754 | |
1755 | For a large project, you would probably have several source files, and use | |
1756 | @code{make} to arrange to recompile them. | |
1757 | ||
1758 | With all the source in a single file, you use the following command to | |
1759 | convert it into a parser file: | |
1760 | ||
1761 | @example | |
fa4d969f | 1762 | bison @var{file}.y |
bfa74976 RS |
1763 | @end example |
1764 | ||
1765 | @noindent | |
1766 | In this example the file was called @file{rpcalc.y} (for ``Reverse Polish | |
fa4d969f | 1767 | @sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c}, |
72d2299c | 1768 | removing the @samp{.y} from the original file name. The file output by |
bfa74976 RS |
1769 | Bison contains the source code for @code{yyparse}. The additional |
1770 | functions in the input file (@code{yylex}, @code{yyerror} and @code{main}) | |
1771 | are copied verbatim to the output. | |
1772 | ||
342b8b6e | 1773 | @node Rpcalc Compile |
bfa74976 RS |
1774 | @subsection Compiling the Parser File |
1775 | @cindex compiling the parser | |
1776 | ||
1777 | Here is how to compile and run the parser file: | |
1778 | ||
1779 | @example | |
1780 | @group | |
1781 | # @r{List files in current directory.} | |
9edcd895 | 1782 | $ @kbd{ls} |
bfa74976 RS |
1783 | rpcalc.tab.c rpcalc.y |
1784 | @end group | |
1785 | ||
1786 | @group | |
1787 | # @r{Compile the Bison parser.} | |
1788 | # @r{@samp{-lm} tells compiler to search math library for @code{pow}.} | |
b56471a6 | 1789 | $ @kbd{cc -lm -o rpcalc rpcalc.tab.c} |
bfa74976 RS |
1790 | @end group |
1791 | ||
1792 | @group | |
1793 | # @r{List files again.} | |
9edcd895 | 1794 | $ @kbd{ls} |
bfa74976 RS |
1795 | rpcalc rpcalc.tab.c rpcalc.y |
1796 | @end group | |
1797 | @end example | |
1798 | ||
1799 | The file @file{rpcalc} now contains the executable code. Here is an | |
1800 | example session using @code{rpcalc}. | |
1801 | ||
1802 | @example | |
9edcd895 AD |
1803 | $ @kbd{rpcalc} |
1804 | @kbd{4 9 +} | |
bfa74976 | 1805 | 13 |
9edcd895 | 1806 | @kbd{3 7 + 3 4 5 *+-} |
bfa74976 | 1807 | -13 |
9edcd895 | 1808 | @kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}} |
bfa74976 | 1809 | 13 |
9edcd895 | 1810 | @kbd{5 6 / 4 n +} |
bfa74976 | 1811 | -3.166666667 |
9edcd895 | 1812 | @kbd{3 4 ^} @r{Exponentiation} |
bfa74976 | 1813 | 81 |
9edcd895 AD |
1814 | @kbd{^D} @r{End-of-file indicator} |
1815 | $ | |
bfa74976 RS |
1816 | @end example |
1817 | ||
342b8b6e | 1818 | @node Infix Calc |
bfa74976 RS |
1819 | @section Infix Notation Calculator: @code{calc} |
1820 | @cindex infix notation calculator | |
1821 | @cindex @code{calc} | |
1822 | @cindex calculator, infix notation | |
1823 | ||
1824 | We now modify rpcalc to handle infix operators instead of postfix. Infix | |
1825 | notation involves the concept of operator precedence and the need for | |
1826 | parentheses nested to arbitrary depth. Here is the Bison code for | |
1827 | @file{calc.y}, an infix desk-top calculator. | |
1828 | ||
1829 | @example | |
38a92d50 | 1830 | /* Infix notation calculator. */ |
bfa74976 RS |
1831 | |
1832 | %@{ | |
38a92d50 PE |
1833 | #define YYSTYPE double |
1834 | #include <math.h> | |
1835 | #include <stdio.h> | |
1836 | int yylex (void); | |
1837 | void yyerror (char const *); | |
bfa74976 RS |
1838 | %@} |
1839 | ||
38a92d50 | 1840 | /* Bison declarations. */ |
bfa74976 RS |
1841 | %token NUM |
1842 | %left '-' '+' | |
1843 | %left '*' '/' | |
1844 | %left NEG /* negation--unary minus */ | |
38a92d50 | 1845 | %right '^' /* exponentiation */ |
bfa74976 | 1846 | |
38a92d50 PE |
1847 | %% /* The grammar follows. */ |
1848 | input: /* empty */ | |
bfa74976 RS |
1849 | | input line |
1850 | ; | |
1851 | ||
1852 | line: '\n' | |
1853 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
1854 | ; | |
1855 | ||
1856 | exp: NUM @{ $$ = $1; @} | |
1857 | | exp '+' exp @{ $$ = $1 + $3; @} | |
1858 | | exp '-' exp @{ $$ = $1 - $3; @} | |
1859 | | exp '*' exp @{ $$ = $1 * $3; @} | |
1860 | | exp '/' exp @{ $$ = $1 / $3; @} | |
1861 | | '-' exp %prec NEG @{ $$ = -$2; @} | |
1862 | | exp '^' exp @{ $$ = pow ($1, $3); @} | |
1863 | | '(' exp ')' @{ $$ = $2; @} | |
1864 | ; | |
1865 | %% | |
1866 | @end example | |
1867 | ||
1868 | @noindent | |
ceed8467 AD |
1869 | The functions @code{yylex}, @code{yyerror} and @code{main} can be the |
1870 | same as before. | |
bfa74976 RS |
1871 | |
1872 | There are two important new features shown in this code. | |
1873 | ||
1874 | In the second section (Bison declarations), @code{%left} declares token | |
1875 | types and says they are left-associative operators. The declarations | |
1876 | @code{%left} and @code{%right} (right associativity) take the place of | |
1877 | @code{%token} which is used to declare a token type name without | |
1878 | associativity. (These tokens are single-character literals, which | |
1879 | ordinarily don't need to be declared. We declare them here to specify | |
1880 | the associativity.) | |
1881 | ||
1882 | Operator precedence is determined by the line ordering of the | |
1883 | declarations; the higher the line number of the declaration (lower on | |
1884 | the page or screen), the higher the precedence. Hence, exponentiation | |
1885 | has the highest precedence, unary minus (@code{NEG}) is next, followed | |
704a47c4 AD |
1886 | by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator |
1887 | Precedence}. | |
bfa74976 | 1888 | |
704a47c4 AD |
1889 | The other important new feature is the @code{%prec} in the grammar |
1890 | section for the unary minus operator. The @code{%prec} simply instructs | |
1891 | Bison that the rule @samp{| '-' exp} has the same precedence as | |
1892 | @code{NEG}---in this case the next-to-highest. @xref{Contextual | |
1893 | Precedence, ,Context-Dependent Precedence}. | |
bfa74976 RS |
1894 | |
1895 | Here is a sample run of @file{calc.y}: | |
1896 | ||
1897 | @need 500 | |
1898 | @example | |
9edcd895 AD |
1899 | $ @kbd{calc} |
1900 | @kbd{4 + 4.5 - (34/(8*3+-3))} | |
bfa74976 | 1901 | 6.880952381 |
9edcd895 | 1902 | @kbd{-56 + 2} |
bfa74976 | 1903 | -54 |
9edcd895 | 1904 | @kbd{3 ^ 2} |
bfa74976 RS |
1905 | 9 |
1906 | @end example | |
1907 | ||
342b8b6e | 1908 | @node Simple Error Recovery |
bfa74976 RS |
1909 | @section Simple Error Recovery |
1910 | @cindex error recovery, simple | |
1911 | ||
1912 | Up to this point, this manual has not addressed the issue of @dfn{error | |
1913 | recovery}---how to continue parsing after the parser detects a syntax | |
ceed8467 AD |
1914 | error. All we have handled is error reporting with @code{yyerror}. |
1915 | Recall that by default @code{yyparse} returns after calling | |
1916 | @code{yyerror}. This means that an erroneous input line causes the | |
1917 | calculator program to exit. Now we show how to rectify this deficiency. | |
bfa74976 RS |
1918 | |
1919 | The Bison language itself includes the reserved word @code{error}, which | |
1920 | may be included in the grammar rules. In the example below it has | |
1921 | been added to one of the alternatives for @code{line}: | |
1922 | ||
1923 | @example | |
1924 | @group | |
1925 | line: '\n' | |
1926 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
1927 | | error '\n' @{ yyerrok; @} | |
1928 | ; | |
1929 | @end group | |
1930 | @end example | |
1931 | ||
ceed8467 | 1932 | This addition to the grammar allows for simple error recovery in the |
6e649e65 | 1933 | event of a syntax error. If an expression that cannot be evaluated is |
ceed8467 AD |
1934 | read, the error will be recognized by the third rule for @code{line}, |
1935 | and parsing will continue. (The @code{yyerror} function is still called | |
1936 | upon to print its message as well.) The action executes the statement | |
1937 | @code{yyerrok}, a macro defined automatically by Bison; its meaning is | |
1938 | that error recovery is complete (@pxref{Error Recovery}). Note the | |
1939 | difference between @code{yyerrok} and @code{yyerror}; neither one is a | |
e0c471a9 | 1940 | misprint. |
bfa74976 RS |
1941 | |
1942 | This form of error recovery deals with syntax errors. There are other | |
1943 | kinds of errors; for example, division by zero, which raises an exception | |
1944 | signal that is normally fatal. A real calculator program must handle this | |
1945 | signal and use @code{longjmp} to return to @code{main} and resume parsing | |
1946 | input lines; it would also have to discard the rest of the current line of | |
1947 | input. We won't discuss this issue further because it is not specific to | |
1948 | Bison programs. | |
1949 | ||
342b8b6e AD |
1950 | @node Location Tracking Calc |
1951 | @section Location Tracking Calculator: @code{ltcalc} | |
1952 | @cindex location tracking calculator | |
1953 | @cindex @code{ltcalc} | |
1954 | @cindex calculator, location tracking | |
1955 | ||
9edcd895 AD |
1956 | This example extends the infix notation calculator with location |
1957 | tracking. This feature will be used to improve the error messages. For | |
1958 | the sake of clarity, this example is a simple integer calculator, since | |
1959 | most of the work needed to use locations will be done in the lexical | |
72d2299c | 1960 | analyzer. |
342b8b6e AD |
1961 | |
1962 | @menu | |
1963 | * Decls: Ltcalc Decls. Bison and C declarations for ltcalc. | |
1964 | * Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations. | |
1965 | * Lexer: Ltcalc Lexer. The lexical analyzer. | |
1966 | @end menu | |
1967 | ||
1968 | @node Ltcalc Decls | |
1969 | @subsection Declarations for @code{ltcalc} | |
1970 | ||
9edcd895 AD |
1971 | The C and Bison declarations for the location tracking calculator are |
1972 | the same as the declarations for the infix notation calculator. | |
342b8b6e AD |
1973 | |
1974 | @example | |
1975 | /* Location tracking calculator. */ | |
1976 | ||
1977 | %@{ | |
38a92d50 PE |
1978 | #define YYSTYPE int |
1979 | #include <math.h> | |
1980 | int yylex (void); | |
1981 | void yyerror (char const *); | |
342b8b6e AD |
1982 | %@} |
1983 | ||
1984 | /* Bison declarations. */ | |
1985 | %token NUM | |
1986 | ||
1987 | %left '-' '+' | |
1988 | %left '*' '/' | |
1989 | %left NEG | |
1990 | %right '^' | |
1991 | ||
38a92d50 | 1992 | %% /* The grammar follows. */ |
342b8b6e AD |
1993 | @end example |
1994 | ||
9edcd895 AD |
1995 | @noindent |
1996 | Note there are no declarations specific to locations. Defining a data | |
1997 | type for storing locations is not needed: we will use the type provided | |
1998 | by default (@pxref{Location Type, ,Data Types of Locations}), which is a | |
1999 | four member structure with the following integer fields: | |
2000 | @code{first_line}, @code{first_column}, @code{last_line} and | |
cd48d21d AD |
2001 | @code{last_column}. By conventions, and in accordance with the GNU |
2002 | Coding Standards and common practice, the line and column count both | |
2003 | start at 1. | |
342b8b6e AD |
2004 | |
2005 | @node Ltcalc Rules | |
2006 | @subsection Grammar Rules for @code{ltcalc} | |
2007 | ||
9edcd895 AD |
2008 | Whether handling locations or not has no effect on the syntax of your |
2009 | language. Therefore, grammar rules for this example will be very close | |
2010 | to those of the previous example: we will only modify them to benefit | |
2011 | from the new information. | |
342b8b6e | 2012 | |
9edcd895 AD |
2013 | Here, we will use locations to report divisions by zero, and locate the |
2014 | wrong expressions or subexpressions. | |
342b8b6e AD |
2015 | |
2016 | @example | |
2017 | @group | |
2018 | input : /* empty */ | |
2019 | | input line | |
2020 | ; | |
2021 | @end group | |
2022 | ||
2023 | @group | |
2024 | line : '\n' | |
2025 | | exp '\n' @{ printf ("%d\n", $1); @} | |
2026 | ; | |
2027 | @end group | |
2028 | ||
2029 | @group | |
2030 | exp : NUM @{ $$ = $1; @} | |
2031 | | exp '+' exp @{ $$ = $1 + $3; @} | |
2032 | | exp '-' exp @{ $$ = $1 - $3; @} | |
2033 | | exp '*' exp @{ $$ = $1 * $3; @} | |
2034 | @end group | |
342b8b6e | 2035 | @group |
9edcd895 | 2036 | | exp '/' exp |
342b8b6e AD |
2037 | @{ |
2038 | if ($3) | |
2039 | $$ = $1 / $3; | |
2040 | else | |
2041 | @{ | |
2042 | $$ = 1; | |
9edcd895 AD |
2043 | fprintf (stderr, "%d.%d-%d.%d: division by zero", |
2044 | @@3.first_line, @@3.first_column, | |
2045 | @@3.last_line, @@3.last_column); | |
342b8b6e AD |
2046 | @} |
2047 | @} | |
2048 | @end group | |
2049 | @group | |
2050 | | '-' exp %preg NEG @{ $$ = -$2; @} | |
2051 | | exp '^' exp @{ $$ = pow ($1, $3); @} | |
2052 | | '(' exp ')' @{ $$ = $2; @} | |
2053 | @end group | |
2054 | @end example | |
2055 | ||
2056 | This code shows how to reach locations inside of semantic actions, by | |
2057 | using the pseudo-variables @code{@@@var{n}} for rule components, and the | |
2058 | pseudo-variable @code{@@$} for groupings. | |
2059 | ||
9edcd895 AD |
2060 | We don't need to assign a value to @code{@@$}: the output parser does it |
2061 | automatically. By default, before executing the C code of each action, | |
2062 | @code{@@$} is set to range from the beginning of @code{@@1} to the end | |
2063 | of @code{@@@var{n}}, for a rule with @var{n} components. This behavior | |
2064 | can be redefined (@pxref{Location Default Action, , Default Action for | |
2065 | Locations}), and for very specific rules, @code{@@$} can be computed by | |
2066 | hand. | |
342b8b6e AD |
2067 | |
2068 | @node Ltcalc Lexer | |
2069 | @subsection The @code{ltcalc} Lexical Analyzer. | |
2070 | ||
9edcd895 | 2071 | Until now, we relied on Bison's defaults to enable location |
72d2299c | 2072 | tracking. The next step is to rewrite the lexical analyzer, and make it |
9edcd895 AD |
2073 | able to feed the parser with the token locations, as it already does for |
2074 | semantic values. | |
342b8b6e | 2075 | |
9edcd895 AD |
2076 | To this end, we must take into account every single character of the |
2077 | input text, to avoid the computed locations of being fuzzy or wrong: | |
342b8b6e AD |
2078 | |
2079 | @example | |
2080 | @group | |
2081 | int | |
2082 | yylex (void) | |
2083 | @{ | |
2084 | int c; | |
18b519c0 | 2085 | @end group |
342b8b6e | 2086 | |
18b519c0 | 2087 | @group |
72d2299c | 2088 | /* Skip white space. */ |
342b8b6e AD |
2089 | while ((c = getchar ()) == ' ' || c == '\t') |
2090 | ++yylloc.last_column; | |
18b519c0 | 2091 | @end group |
342b8b6e | 2092 | |
18b519c0 | 2093 | @group |
72d2299c | 2094 | /* Step. */ |
342b8b6e AD |
2095 | yylloc.first_line = yylloc.last_line; |
2096 | yylloc.first_column = yylloc.last_column; | |
2097 | @end group | |
2098 | ||
2099 | @group | |
72d2299c | 2100 | /* Process numbers. */ |
342b8b6e AD |
2101 | if (isdigit (c)) |
2102 | @{ | |
2103 | yylval = c - '0'; | |
2104 | ++yylloc.last_column; | |
2105 | while (isdigit (c = getchar ())) | |
2106 | @{ | |
2107 | ++yylloc.last_column; | |
2108 | yylval = yylval * 10 + c - '0'; | |
2109 | @} | |
2110 | ungetc (c, stdin); | |
2111 | return NUM; | |
2112 | @} | |
2113 | @end group | |
2114 | ||
72d2299c | 2115 | /* Return end-of-input. */ |
342b8b6e AD |
2116 | if (c == EOF) |
2117 | return 0; | |
2118 | ||
72d2299c | 2119 | /* Return a single char, and update location. */ |
342b8b6e AD |
2120 | if (c == '\n') |
2121 | @{ | |
2122 | ++yylloc.last_line; | |
2123 | yylloc.last_column = 0; | |
2124 | @} | |
2125 | else | |
2126 | ++yylloc.last_column; | |
2127 | return c; | |
2128 | @} | |
2129 | @end example | |
2130 | ||
9edcd895 AD |
2131 | Basically, the lexical analyzer performs the same processing as before: |
2132 | it skips blanks and tabs, and reads numbers or single-character tokens. | |
2133 | In addition, it updates @code{yylloc}, the global variable (of type | |
2134 | @code{YYLTYPE}) containing the token's location. | |
342b8b6e | 2135 | |
9edcd895 | 2136 | Now, each time this function returns a token, the parser has its number |
72d2299c | 2137 | as well as its semantic value, and its location in the text. The last |
9edcd895 AD |
2138 | needed change is to initialize @code{yylloc}, for example in the |
2139 | controlling function: | |
342b8b6e AD |
2140 | |
2141 | @example | |
9edcd895 | 2142 | @group |
342b8b6e AD |
2143 | int |
2144 | main (void) | |
2145 | @{ | |
2146 | yylloc.first_line = yylloc.last_line = 1; | |
2147 | yylloc.first_column = yylloc.last_column = 0; | |
2148 | return yyparse (); | |
2149 | @} | |
9edcd895 | 2150 | @end group |
342b8b6e AD |
2151 | @end example |
2152 | ||
9edcd895 AD |
2153 | Remember that computing locations is not a matter of syntax. Every |
2154 | character must be associated to a location update, whether it is in | |
2155 | valid input, in comments, in literal strings, and so on. | |
342b8b6e AD |
2156 | |
2157 | @node Multi-function Calc | |
bfa74976 RS |
2158 | @section Multi-Function Calculator: @code{mfcalc} |
2159 | @cindex multi-function calculator | |
2160 | @cindex @code{mfcalc} | |
2161 | @cindex calculator, multi-function | |
2162 | ||
2163 | Now that the basics of Bison have been discussed, it is time to move on to | |
2164 | a more advanced problem. The above calculators provided only five | |
2165 | functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would | |
2166 | be nice to have a calculator that provides other mathematical functions such | |
2167 | as @code{sin}, @code{cos}, etc. | |
2168 | ||
2169 | It is easy to add new operators to the infix calculator as long as they are | |
2170 | only single-character literals. The lexical analyzer @code{yylex} passes | |
9d9b8b70 | 2171 | back all nonnumeric characters as tokens, so new grammar rules suffice for |
bfa74976 RS |
2172 | adding a new operator. But we want something more flexible: built-in |
2173 | functions whose syntax has this form: | |
2174 | ||
2175 | @example | |
2176 | @var{function_name} (@var{argument}) | |
2177 | @end example | |
2178 | ||
2179 | @noindent | |
2180 | At the same time, we will add memory to the calculator, by allowing you | |
2181 | to create named variables, store values in them, and use them later. | |
2182 | Here is a sample session with the multi-function calculator: | |
2183 | ||
2184 | @example | |
9edcd895 AD |
2185 | $ @kbd{mfcalc} |
2186 | @kbd{pi = 3.141592653589} | |
bfa74976 | 2187 | 3.1415926536 |
9edcd895 | 2188 | @kbd{sin(pi)} |
bfa74976 | 2189 | 0.0000000000 |
9edcd895 | 2190 | @kbd{alpha = beta1 = 2.3} |
bfa74976 | 2191 | 2.3000000000 |
9edcd895 | 2192 | @kbd{alpha} |
bfa74976 | 2193 | 2.3000000000 |
9edcd895 | 2194 | @kbd{ln(alpha)} |
bfa74976 | 2195 | 0.8329091229 |
9edcd895 | 2196 | @kbd{exp(ln(beta1))} |
bfa74976 | 2197 | 2.3000000000 |
9edcd895 | 2198 | $ |
bfa74976 RS |
2199 | @end example |
2200 | ||
2201 | Note that multiple assignment and nested function calls are permitted. | |
2202 | ||
2203 | @menu | |
2204 | * Decl: Mfcalc Decl. Bison declarations for multi-function calculator. | |
2205 | * Rules: Mfcalc Rules. Grammar rules for the calculator. | |
2206 | * Symtab: Mfcalc Symtab. Symbol table management subroutines. | |
2207 | @end menu | |
2208 | ||
342b8b6e | 2209 | @node Mfcalc Decl |
bfa74976 RS |
2210 | @subsection Declarations for @code{mfcalc} |
2211 | ||
2212 | Here are the C and Bison declarations for the multi-function calculator. | |
2213 | ||
2214 | @smallexample | |
18b519c0 | 2215 | @group |
bfa74976 | 2216 | %@{ |
38a92d50 PE |
2217 | #include <math.h> /* For math functions, cos(), sin(), etc. */ |
2218 | #include "calc.h" /* Contains definition of `symrec'. */ | |
2219 | int yylex (void); | |
2220 | void yyerror (char const *); | |
bfa74976 | 2221 | %@} |
18b519c0 AD |
2222 | @end group |
2223 | @group | |
bfa74976 | 2224 | %union @{ |
38a92d50 PE |
2225 | double val; /* For returning numbers. */ |
2226 | symrec *tptr; /* For returning symbol-table pointers. */ | |
bfa74976 | 2227 | @} |
18b519c0 | 2228 | @end group |
38a92d50 PE |
2229 | %token <val> NUM /* Simple double precision number. */ |
2230 | %token <tptr> VAR FNCT /* Variable and Function. */ | |
bfa74976 RS |
2231 | %type <val> exp |
2232 | ||
18b519c0 | 2233 | @group |
bfa74976 RS |
2234 | %right '=' |
2235 | %left '-' '+' | |
2236 | %left '*' '/' | |
38a92d50 PE |
2237 | %left NEG /* negation--unary minus */ |
2238 | %right '^' /* exponentiation */ | |
18b519c0 | 2239 | @end group |
38a92d50 | 2240 | %% /* The grammar follows. */ |
bfa74976 RS |
2241 | @end smallexample |
2242 | ||
2243 | The above grammar introduces only two new features of the Bison language. | |
2244 | These features allow semantic values to have various data types | |
2245 | (@pxref{Multiple Types, ,More Than One Value Type}). | |
2246 | ||
2247 | The @code{%union} declaration specifies the entire list of possible types; | |
2248 | this is instead of defining @code{YYSTYPE}. The allowable types are now | |
2249 | double-floats (for @code{exp} and @code{NUM}) and pointers to entries in | |
2250 | the symbol table. @xref{Union Decl, ,The Collection of Value Types}. | |
2251 | ||
2252 | Since values can now have various types, it is necessary to associate a | |
2253 | type with each grammar symbol whose semantic value is used. These symbols | |
2254 | are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their | |
2255 | declarations are augmented with information about their data type (placed | |
2256 | between angle brackets). | |
2257 | ||
704a47c4 AD |
2258 | The Bison construct @code{%type} is used for declaring nonterminal |
2259 | symbols, just as @code{%token} is used for declaring token types. We | |
2260 | have not used @code{%type} before because nonterminal symbols are | |
2261 | normally declared implicitly by the rules that define them. But | |
2262 | @code{exp} must be declared explicitly so we can specify its value type. | |
2263 | @xref{Type Decl, ,Nonterminal Symbols}. | |
bfa74976 | 2264 | |
342b8b6e | 2265 | @node Mfcalc Rules |
bfa74976 RS |
2266 | @subsection Grammar Rules for @code{mfcalc} |
2267 | ||
2268 | Here are the grammar rules for the multi-function calculator. | |
2269 | Most of them are copied directly from @code{calc}; three rules, | |
2270 | those which mention @code{VAR} or @code{FNCT}, are new. | |
2271 | ||
2272 | @smallexample | |
18b519c0 | 2273 | @group |
bfa74976 RS |
2274 | input: /* empty */ |
2275 | | input line | |
2276 | ; | |
18b519c0 | 2277 | @end group |
bfa74976 | 2278 | |
18b519c0 | 2279 | @group |
bfa74976 RS |
2280 | line: |
2281 | '\n' | |
2282 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
2283 | | error '\n' @{ yyerrok; @} | |
2284 | ; | |
18b519c0 | 2285 | @end group |
bfa74976 | 2286 | |
18b519c0 | 2287 | @group |
bfa74976 RS |
2288 | exp: NUM @{ $$ = $1; @} |
2289 | | VAR @{ $$ = $1->value.var; @} | |
2290 | | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @} | |
2291 | | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @} | |
2292 | | exp '+' exp @{ $$ = $1 + $3; @} | |
2293 | | exp '-' exp @{ $$ = $1 - $3; @} | |
2294 | | exp '*' exp @{ $$ = $1 * $3; @} | |
2295 | | exp '/' exp @{ $$ = $1 / $3; @} | |
2296 | | '-' exp %prec NEG @{ $$ = -$2; @} | |
2297 | | exp '^' exp @{ $$ = pow ($1, $3); @} | |
2298 | | '(' exp ')' @{ $$ = $2; @} | |
2299 | ; | |
18b519c0 | 2300 | @end group |
38a92d50 | 2301 | /* End of grammar. */ |
bfa74976 RS |
2302 | %% |
2303 | @end smallexample | |
2304 | ||
342b8b6e | 2305 | @node Mfcalc Symtab |
bfa74976 RS |
2306 | @subsection The @code{mfcalc} Symbol Table |
2307 | @cindex symbol table example | |
2308 | ||
2309 | The multi-function calculator requires a symbol table to keep track of the | |
2310 | names and meanings of variables and functions. This doesn't affect the | |
2311 | grammar rules (except for the actions) or the Bison declarations, but it | |
2312 | requires some additional C functions for support. | |
2313 | ||
2314 | The symbol table itself consists of a linked list of records. Its | |
2315 | definition, which is kept in the header @file{calc.h}, is as follows. It | |
2316 | provides for either functions or variables to be placed in the table. | |
2317 | ||
2318 | @smallexample | |
2319 | @group | |
38a92d50 | 2320 | /* Function type. */ |
32dfccf8 | 2321 | typedef double (*func_t) (double); |
72f889cc | 2322 | @end group |
32dfccf8 | 2323 | |
72f889cc | 2324 | @group |
38a92d50 | 2325 | /* Data type for links in the chain of symbols. */ |
bfa74976 RS |
2326 | struct symrec |
2327 | @{ | |
38a92d50 | 2328 | char *name; /* name of symbol */ |
bfa74976 | 2329 | int type; /* type of symbol: either VAR or FNCT */ |
32dfccf8 AD |
2330 | union |
2331 | @{ | |
38a92d50 PE |
2332 | double var; /* value of a VAR */ |
2333 | func_t fnctptr; /* value of a FNCT */ | |
bfa74976 | 2334 | @} value; |
38a92d50 | 2335 | struct symrec *next; /* link field */ |
bfa74976 RS |
2336 | @}; |
2337 | @end group | |
2338 | ||
2339 | @group | |
2340 | typedef struct symrec symrec; | |
2341 | ||
38a92d50 | 2342 | /* The symbol table: a chain of `struct symrec'. */ |
bfa74976 RS |
2343 | extern symrec *sym_table; |
2344 | ||
a730d142 | 2345 | symrec *putsym (char const *, int); |
38a92d50 | 2346 | symrec *getsym (char const *); |
bfa74976 RS |
2347 | @end group |
2348 | @end smallexample | |
2349 | ||
2350 | The new version of @code{main} includes a call to @code{init_table}, a | |
2351 | function that initializes the symbol table. Here it is, and | |
2352 | @code{init_table} as well: | |
2353 | ||
2354 | @smallexample | |
bfa74976 RS |
2355 | #include <stdio.h> |
2356 | ||
18b519c0 | 2357 | @group |
38a92d50 | 2358 | /* Called by yyparse on error. */ |
13863333 | 2359 | void |
38a92d50 | 2360 | yyerror (char const *s) |
bfa74976 RS |
2361 | @{ |
2362 | printf ("%s\n", s); | |
2363 | @} | |
18b519c0 | 2364 | @end group |
bfa74976 | 2365 | |
18b519c0 | 2366 | @group |
bfa74976 RS |
2367 | struct init |
2368 | @{ | |
38a92d50 PE |
2369 | char const *fname; |
2370 | double (*fnct) (double); | |
bfa74976 RS |
2371 | @}; |
2372 | @end group | |
2373 | ||
2374 | @group | |
38a92d50 | 2375 | struct init const arith_fncts[] = |
13863333 | 2376 | @{ |
32dfccf8 AD |
2377 | "sin", sin, |
2378 | "cos", cos, | |
13863333 | 2379 | "atan", atan, |
32dfccf8 AD |
2380 | "ln", log, |
2381 | "exp", exp, | |
13863333 AD |
2382 | "sqrt", sqrt, |
2383 | 0, 0 | |
2384 | @}; | |
18b519c0 | 2385 | @end group |
bfa74976 | 2386 | |
18b519c0 | 2387 | @group |
bfa74976 | 2388 | /* The symbol table: a chain of `struct symrec'. */ |
38a92d50 | 2389 | symrec *sym_table; |
bfa74976 RS |
2390 | @end group |
2391 | ||
2392 | @group | |
72d2299c | 2393 | /* Put arithmetic functions in table. */ |
13863333 AD |
2394 | void |
2395 | init_table (void) | |
bfa74976 RS |
2396 | @{ |
2397 | int i; | |
2398 | symrec *ptr; | |
2399 | for (i = 0; arith_fncts[i].fname != 0; i++) | |
2400 | @{ | |
2401 | ptr = putsym (arith_fncts[i].fname, FNCT); | |
2402 | ptr->value.fnctptr = arith_fncts[i].fnct; | |
2403 | @} | |
2404 | @} | |
2405 | @end group | |
38a92d50 PE |
2406 | |
2407 | @group | |
2408 | int | |
2409 | main (void) | |
2410 | @{ | |
2411 | init_table (); | |
2412 | return yyparse (); | |
2413 | @} | |
2414 | @end group | |
bfa74976 RS |
2415 | @end smallexample |
2416 | ||
2417 | By simply editing the initialization list and adding the necessary include | |
2418 | files, you can add additional functions to the calculator. | |
2419 | ||
2420 | Two important functions allow look-up and installation of symbols in the | |
2421 | symbol table. The function @code{putsym} is passed a name and the type | |
2422 | (@code{VAR} or @code{FNCT}) of the object to be installed. The object is | |
2423 | linked to the front of the list, and a pointer to the object is returned. | |
2424 | The function @code{getsym} is passed the name of the symbol to look up. If | |
2425 | found, a pointer to that symbol is returned; otherwise zero is returned. | |
2426 | ||
2427 | @smallexample | |
2428 | symrec * | |
38a92d50 | 2429 | putsym (char const *sym_name, int sym_type) |
bfa74976 RS |
2430 | @{ |
2431 | symrec *ptr; | |
2432 | ptr = (symrec *) malloc (sizeof (symrec)); | |
2433 | ptr->name = (char *) malloc (strlen (sym_name) + 1); | |
2434 | strcpy (ptr->name,sym_name); | |
2435 | ptr->type = sym_type; | |
72d2299c | 2436 | ptr->value.var = 0; /* Set value to 0 even if fctn. */ |
bfa74976 RS |
2437 | ptr->next = (struct symrec *)sym_table; |
2438 | sym_table = ptr; | |
2439 | return ptr; | |
2440 | @} | |
2441 | ||
2442 | symrec * | |
38a92d50 | 2443 | getsym (char const *sym_name) |
bfa74976 RS |
2444 | @{ |
2445 | symrec *ptr; | |
2446 | for (ptr = sym_table; ptr != (symrec *) 0; | |
2447 | ptr = (symrec *)ptr->next) | |
2448 | if (strcmp (ptr->name,sym_name) == 0) | |
2449 | return ptr; | |
2450 | return 0; | |
2451 | @} | |
2452 | @end smallexample | |
2453 | ||
2454 | The function @code{yylex} must now recognize variables, numeric values, and | |
2455 | the single-character arithmetic operators. Strings of alphanumeric | |
9d9b8b70 | 2456 | characters with a leading letter are recognized as either variables or |
bfa74976 RS |
2457 | functions depending on what the symbol table says about them. |
2458 | ||
2459 | The string is passed to @code{getsym} for look up in the symbol table. If | |
2460 | the name appears in the table, a pointer to its location and its type | |
2461 | (@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not | |
2462 | already in the table, then it is installed as a @code{VAR} using | |
2463 | @code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is | |
e0c471a9 | 2464 | returned to @code{yyparse}. |
bfa74976 RS |
2465 | |
2466 | No change is needed in the handling of numeric values and arithmetic | |
2467 | operators in @code{yylex}. | |
2468 | ||
2469 | @smallexample | |
2470 | @group | |
2471 | #include <ctype.h> | |
18b519c0 | 2472 | @end group |
13863333 | 2473 | |
18b519c0 | 2474 | @group |
13863333 AD |
2475 | int |
2476 | yylex (void) | |
bfa74976 RS |
2477 | @{ |
2478 | int c; | |
2479 | ||
72d2299c | 2480 | /* Ignore white space, get first nonwhite character. */ |
bfa74976 RS |
2481 | while ((c = getchar ()) == ' ' || c == '\t'); |
2482 | ||
2483 | if (c == EOF) | |
2484 | return 0; | |
2485 | @end group | |
2486 | ||
2487 | @group | |
2488 | /* Char starts a number => parse the number. */ | |
2489 | if (c == '.' || isdigit (c)) | |
2490 | @{ | |
2491 | ungetc (c, stdin); | |
2492 | scanf ("%lf", &yylval.val); | |
2493 | return NUM; | |
2494 | @} | |
2495 | @end group | |
2496 | ||
2497 | @group | |
2498 | /* Char starts an identifier => read the name. */ | |
2499 | if (isalpha (c)) | |
2500 | @{ | |
2501 | symrec *s; | |
2502 | static char *symbuf = 0; | |
2503 | static int length = 0; | |
2504 | int i; | |
2505 | @end group | |
2506 | ||
2507 | @group | |
2508 | /* Initially make the buffer long enough | |
2509 | for a 40-character symbol name. */ | |
2510 | if (length == 0) | |
2511 | length = 40, symbuf = (char *)malloc (length + 1); | |
2512 | ||
2513 | i = 0; | |
2514 | do | |
2515 | @end group | |
2516 | @group | |
2517 | @{ | |
2518 | /* If buffer is full, make it bigger. */ | |
2519 | if (i == length) | |
2520 | @{ | |
2521 | length *= 2; | |
18b519c0 | 2522 | symbuf = (char *) realloc (symbuf, length + 1); |
bfa74976 RS |
2523 | @} |
2524 | /* Add this character to the buffer. */ | |
2525 | symbuf[i++] = c; | |
2526 | /* Get another character. */ | |
2527 | c = getchar (); | |
2528 | @} | |
2529 | @end group | |
2530 | @group | |
72d2299c | 2531 | while (isalnum (c)); |
bfa74976 RS |
2532 | |
2533 | ungetc (c, stdin); | |
2534 | symbuf[i] = '\0'; | |
2535 | @end group | |
2536 | ||
2537 | @group | |
2538 | s = getsym (symbuf); | |
2539 | if (s == 0) | |
2540 | s = putsym (symbuf, VAR); | |
2541 | yylval.tptr = s; | |
2542 | return s->type; | |
2543 | @} | |
2544 | ||
2545 | /* Any other character is a token by itself. */ | |
2546 | return c; | |
2547 | @} | |
2548 | @end group | |
2549 | @end smallexample | |
2550 | ||
72d2299c | 2551 | This program is both powerful and flexible. You may easily add new |
704a47c4 AD |
2552 | functions, and it is a simple job to modify this code to install |
2553 | predefined variables such as @code{pi} or @code{e} as well. | |
bfa74976 | 2554 | |
342b8b6e | 2555 | @node Exercises |
bfa74976 RS |
2556 | @section Exercises |
2557 | @cindex exercises | |
2558 | ||
2559 | @enumerate | |
2560 | @item | |
2561 | Add some new functions from @file{math.h} to the initialization list. | |
2562 | ||
2563 | @item | |
2564 | Add another array that contains constants and their values. Then | |
2565 | modify @code{init_table} to add these constants to the symbol table. | |
2566 | It will be easiest to give the constants type @code{VAR}. | |
2567 | ||
2568 | @item | |
2569 | Make the program report an error if the user refers to an | |
2570 | uninitialized variable in any way except to store a value in it. | |
2571 | @end enumerate | |
2572 | ||
342b8b6e | 2573 | @node Grammar File |
bfa74976 RS |
2574 | @chapter Bison Grammar Files |
2575 | ||
2576 | Bison takes as input a context-free grammar specification and produces a | |
2577 | C-language function that recognizes correct instances of the grammar. | |
2578 | ||
2579 | The Bison grammar input file conventionally has a name ending in @samp{.y}. | |
234a3be3 | 2580 | @xref{Invocation, ,Invoking Bison}. |
bfa74976 RS |
2581 | |
2582 | @menu | |
2583 | * Grammar Outline:: Overall layout of the grammar file. | |
2584 | * Symbols:: Terminal and nonterminal symbols. | |
2585 | * Rules:: How to write grammar rules. | |
2586 | * Recursion:: Writing recursive rules. | |
2587 | * Semantics:: Semantic values and actions. | |
847bf1f5 | 2588 | * Locations:: Locations and actions. |
bfa74976 RS |
2589 | * Declarations:: All kinds of Bison declarations are described here. |
2590 | * Multiple Parsers:: Putting more than one Bison parser in one program. | |
2591 | @end menu | |
2592 | ||
342b8b6e | 2593 | @node Grammar Outline |
bfa74976 RS |
2594 | @section Outline of a Bison Grammar |
2595 | ||
2596 | A Bison grammar file has four main sections, shown here with the | |
2597 | appropriate delimiters: | |
2598 | ||
2599 | @example | |
2600 | %@{ | |
38a92d50 | 2601 | @var{Prologue} |
bfa74976 RS |
2602 | %@} |
2603 | ||
2604 | @var{Bison declarations} | |
2605 | ||
2606 | %% | |
2607 | @var{Grammar rules} | |
2608 | %% | |
2609 | ||
75f5aaea | 2610 | @var{Epilogue} |
bfa74976 RS |
2611 | @end example |
2612 | ||
2613 | Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections. | |
2bfc2e2a PE |
2614 | As a @acronym{GNU} extension, @samp{//} introduces a comment that |
2615 | continues until end of line. | |
bfa74976 RS |
2616 | |
2617 | @menu | |
75f5aaea | 2618 | * Prologue:: Syntax and usage of the prologue. |
bfa74976 RS |
2619 | * Bison Declarations:: Syntax and usage of the Bison declarations section. |
2620 | * Grammar Rules:: Syntax and usage of the grammar rules section. | |
75f5aaea | 2621 | * Epilogue:: Syntax and usage of the epilogue. |
bfa74976 RS |
2622 | @end menu |
2623 | ||
38a92d50 | 2624 | @node Prologue |
75f5aaea MA |
2625 | @subsection The prologue |
2626 | @cindex declarations section | |
2627 | @cindex Prologue | |
2628 | @cindex declarations | |
bfa74976 | 2629 | |
f8e1c9e5 AD |
2630 | The @var{Prologue} section contains macro definitions and declarations |
2631 | of functions and variables that are used in the actions in the grammar | |
2632 | rules. These are copied to the beginning of the parser file so that | |
2633 | they precede the definition of @code{yyparse}. You can use | |
2634 | @samp{#include} to get the declarations from a header file. If you | |
2635 | don't need any C declarations, you may omit the @samp{%@{} and | |
2636 | @samp{%@}} delimiters that bracket this section. | |
bfa74976 | 2637 | |
287c78f6 PE |
2638 | The @var{Prologue} section is terminated by the the first occurrence |
2639 | of @samp{%@}} that is outside a comment, a string literal, or a | |
2640 | character constant. | |
2641 | ||
c732d2c6 AD |
2642 | You may have more than one @var{Prologue} section, intermixed with the |
2643 | @var{Bison declarations}. This allows you to have C and Bison | |
2644 | declarations that refer to each other. For example, the @code{%union} | |
2645 | declaration may use types defined in a header file, and you may wish to | |
2646 | prototype functions that take arguments of type @code{YYSTYPE}. This | |
2647 | can be done with two @var{Prologue} blocks, one before and one after the | |
2648 | @code{%union} declaration. | |
2649 | ||
2650 | @smallexample | |
2651 | %@{ | |
38a92d50 PE |
2652 | #include <stdio.h> |
2653 | #include "ptypes.h" | |
c732d2c6 AD |
2654 | %@} |
2655 | ||
2656 | %union @{ | |
779e7ceb | 2657 | long int n; |
c732d2c6 AD |
2658 | tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */ |
2659 | @} | |
2660 | ||
2661 | %@{ | |
38a92d50 PE |
2662 | static void print_token_value (FILE *, int, YYSTYPE); |
2663 | #define YYPRINT(F, N, L) print_token_value (F, N, L) | |
c732d2c6 AD |
2664 | %@} |
2665 | ||
2666 | @dots{} | |
2667 | @end smallexample | |
2668 | ||
34f98f46 JD |
2669 | @findex %before-header |
2670 | @findex %start-header | |
2671 | @findex %after-header | |
9bc0dd67 JD |
2672 | If you've instructed Bison to generate a header file (@pxref{Table of Symbols, |
2673 | ,%defines}), you probably want @code{#include "ptypes.h"} to appear | |
2674 | in that header file as well. | |
34f98f46 JD |
2675 | In that case, use @code{%before-header}, @code{%start-header}, and |
2676 | @code{%after-header} instead of @var{Prologue} sections | |
2677 | (@pxref{Table of Symbols, ,%start-header}): | |
9bc0dd67 JD |
2678 | |
2679 | @smallexample | |
34f98f46 | 2680 | %before-header @{ |
9bc0dd67 | 2681 | #include <stdio.h> |
34f98f46 | 2682 | @} |
9bc0dd67 | 2683 | |
34f98f46 | 2684 | %start-header @{ |
9bc0dd67 JD |
2685 | #include "ptypes.h" |
2686 | @} | |
2687 | %union @{ | |
2688 | long int n; | |
2689 | tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */ | |
2690 | @} | |
2691 | ||
34f98f46 | 2692 | %after-header @{ |
9bc0dd67 JD |
2693 | static void print_token_value (FILE *, int, YYSTYPE); |
2694 | #define YYPRINT(F, N, L) print_token_value (F, N, L) | |
34f98f46 | 2695 | @} |
9bc0dd67 JD |
2696 | |
2697 | @dots{} | |
2698 | @end smallexample | |
2699 | ||
342b8b6e | 2700 | @node Bison Declarations |
bfa74976 RS |
2701 | @subsection The Bison Declarations Section |
2702 | @cindex Bison declarations (introduction) | |
2703 | @cindex declarations, Bison (introduction) | |
2704 | ||
2705 | The @var{Bison declarations} section contains declarations that define | |
2706 | terminal and nonterminal symbols, specify precedence, and so on. | |
2707 | In some simple grammars you may not need any declarations. | |
2708 | @xref{Declarations, ,Bison Declarations}. | |
2709 | ||
342b8b6e | 2710 | @node Grammar Rules |
bfa74976 RS |
2711 | @subsection The Grammar Rules Section |
2712 | @cindex grammar rules section | |
2713 | @cindex rules section for grammar | |
2714 | ||
2715 | The @dfn{grammar rules} section contains one or more Bison grammar | |
2716 | rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}. | |
2717 | ||
2718 | There must always be at least one grammar rule, and the first | |
2719 | @samp{%%} (which precedes the grammar rules) may never be omitted even | |
2720 | if it is the first thing in the file. | |
2721 | ||
38a92d50 | 2722 | @node Epilogue |
75f5aaea | 2723 | @subsection The epilogue |
bfa74976 | 2724 | @cindex additional C code section |
75f5aaea | 2725 | @cindex epilogue |
bfa74976 RS |
2726 | @cindex C code, section for additional |
2727 | ||
08e49d20 PE |
2728 | The @var{Epilogue} is copied verbatim to the end of the parser file, just as |
2729 | the @var{Prologue} is copied to the beginning. This is the most convenient | |
342b8b6e AD |
2730 | place to put anything that you want to have in the parser file but which need |
2731 | not come before the definition of @code{yyparse}. For example, the | |
38a92d50 PE |
2732 | definitions of @code{yylex} and @code{yyerror} often go here. Because |
2733 | C requires functions to be declared before being used, you often need | |
2734 | to declare functions like @code{yylex} and @code{yyerror} in the Prologue, | |
e4f85c39 | 2735 | even if you define them in the Epilogue. |
75f5aaea | 2736 | @xref{Interface, ,Parser C-Language Interface}. |
bfa74976 RS |
2737 | |
2738 | If the last section is empty, you may omit the @samp{%%} that separates it | |
2739 | from the grammar rules. | |
2740 | ||
f8e1c9e5 AD |
2741 | The Bison parser itself contains many macros and identifiers whose names |
2742 | start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using | |
2743 | any such names (except those documented in this manual) in the epilogue | |
2744 | of the grammar file. | |
bfa74976 | 2745 | |
342b8b6e | 2746 | @node Symbols |
bfa74976 RS |
2747 | @section Symbols, Terminal and Nonterminal |
2748 | @cindex nonterminal symbol | |
2749 | @cindex terminal symbol | |
2750 | @cindex token type | |
2751 | @cindex symbol | |
2752 | ||
2753 | @dfn{Symbols} in Bison grammars represent the grammatical classifications | |
2754 | of the language. | |
2755 | ||
2756 | A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a | |
2757 | class of syntactically equivalent tokens. You use the symbol in grammar | |
2758 | rules to mean that a token in that class is allowed. The symbol is | |
2759 | represented in the Bison parser by a numeric code, and the @code{yylex} | |
f8e1c9e5 AD |
2760 | function returns a token type code to indicate what kind of token has |
2761 | been read. You don't need to know what the code value is; you can use | |
2762 | the symbol to stand for it. | |
bfa74976 | 2763 | |
f8e1c9e5 AD |
2764 | A @dfn{nonterminal symbol} stands for a class of syntactically |
2765 | equivalent groupings. The symbol name is used in writing grammar rules. | |
2766 | By convention, it should be all lower case. | |
bfa74976 RS |
2767 | |
2768 | Symbol names can contain letters, digits (not at the beginning), | |
2769 | underscores and periods. Periods make sense only in nonterminals. | |
2770 | ||
931c7513 | 2771 | There are three ways of writing terminal symbols in the grammar: |
bfa74976 RS |
2772 | |
2773 | @itemize @bullet | |
2774 | @item | |
2775 | A @dfn{named token type} is written with an identifier, like an | |
c827f760 | 2776 | identifier in C@. By convention, it should be all upper case. Each |
bfa74976 RS |
2777 | such name must be defined with a Bison declaration such as |
2778 | @code{%token}. @xref{Token Decl, ,Token Type Names}. | |
2779 | ||
2780 | @item | |
2781 | @cindex character token | |
2782 | @cindex literal token | |
2783 | @cindex single-character literal | |
931c7513 RS |
2784 | A @dfn{character token type} (or @dfn{literal character token}) is |
2785 | written in the grammar using the same syntax used in C for character | |
2786 | constants; for example, @code{'+'} is a character token type. A | |
2787 | character token type doesn't need to be declared unless you need to | |
2788 | specify its semantic value data type (@pxref{Value Type, ,Data Types of | |
2789 | Semantic Values}), associativity, or precedence (@pxref{Precedence, | |
2790 | ,Operator Precedence}). | |
bfa74976 RS |
2791 | |
2792 | By convention, a character token type is used only to represent a | |
2793 | token that consists of that particular character. Thus, the token | |
2794 | type @code{'+'} is used to represent the character @samp{+} as a | |
2795 | token. Nothing enforces this convention, but if you depart from it, | |
2796 | your program will confuse other readers. | |
2797 | ||
2798 | All the usual escape sequences used in character literals in C can be | |
2799 | used in Bison as well, but you must not use the null character as a | |
72d2299c PE |
2800 | character literal because its numeric code, zero, signifies |
2801 | end-of-input (@pxref{Calling Convention, ,Calling Convention | |
2bfc2e2a PE |
2802 | for @code{yylex}}). Also, unlike standard C, trigraphs have no |
2803 | special meaning in Bison character literals, nor is backslash-newline | |
2804 | allowed. | |
931c7513 RS |
2805 | |
2806 | @item | |
2807 | @cindex string token | |
2808 | @cindex literal string token | |
9ecbd125 | 2809 | @cindex multicharacter literal |
931c7513 RS |
2810 | A @dfn{literal string token} is written like a C string constant; for |
2811 | example, @code{"<="} is a literal string token. A literal string token | |
2812 | doesn't need to be declared unless you need to specify its semantic | |
14ded682 | 2813 | value data type (@pxref{Value Type}), associativity, or precedence |
931c7513 RS |
2814 | (@pxref{Precedence}). |
2815 | ||
2816 | You can associate the literal string token with a symbolic name as an | |
2817 | alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token | |
2818 | Declarations}). If you don't do that, the lexical analyzer has to | |
2819 | retrieve the token number for the literal string token from the | |
2820 | @code{yytname} table (@pxref{Calling Convention}). | |
2821 | ||
c827f760 | 2822 | @strong{Warning}: literal string tokens do not work in Yacc. |
931c7513 RS |
2823 | |
2824 | By convention, a literal string token is used only to represent a token | |
2825 | that consists of that particular string. Thus, you should use the token | |
2826 | type @code{"<="} to represent the string @samp{<=} as a token. Bison | |
9ecbd125 | 2827 | does not enforce this convention, but if you depart from it, people who |
931c7513 RS |
2828 | read your program will be confused. |
2829 | ||
2830 | All the escape sequences used in string literals in C can be used in | |
92ac3705 PE |
2831 | Bison as well, except that you must not use a null character within a |
2832 | string literal. Also, unlike Standard C, trigraphs have no special | |
2bfc2e2a PE |
2833 | meaning in Bison string literals, nor is backslash-newline allowed. A |
2834 | literal string token must contain two or more characters; for a token | |
2835 | containing just one character, use a character token (see above). | |
bfa74976 RS |
2836 | @end itemize |
2837 | ||
2838 | How you choose to write a terminal symbol has no effect on its | |
2839 | grammatical meaning. That depends only on where it appears in rules and | |
2840 | on when the parser function returns that symbol. | |
2841 | ||
72d2299c PE |
2842 | The value returned by @code{yylex} is always one of the terminal |
2843 | symbols, except that a zero or negative value signifies end-of-input. | |
2844 | Whichever way you write the token type in the grammar rules, you write | |
2845 | it the same way in the definition of @code{yylex}. The numeric code | |
2846 | for a character token type is simply the positive numeric code of the | |
2847 | character, so @code{yylex} can use the identical value to generate the | |
2848 | requisite code, though you may need to convert it to @code{unsigned | |
2849 | char} to avoid sign-extension on hosts where @code{char} is signed. | |
2850 | Each named token type becomes a C macro in | |
bfa74976 | 2851 | the parser file, so @code{yylex} can use the name to stand for the code. |
13863333 | 2852 | (This is why periods don't make sense in terminal symbols.) |
bfa74976 RS |
2853 | @xref{Calling Convention, ,Calling Convention for @code{yylex}}. |
2854 | ||
2855 | If @code{yylex} is defined in a separate file, you need to arrange for the | |
2856 | token-type macro definitions to be available there. Use the @samp{-d} | |
2857 | option when you run Bison, so that it will write these macro definitions | |
2858 | into a separate header file @file{@var{name}.tab.h} which you can include | |
2859 | in the other source files that need it. @xref{Invocation, ,Invoking Bison}. | |
2860 | ||
72d2299c | 2861 | If you want to write a grammar that is portable to any Standard C |
9d9b8b70 | 2862 | host, you must use only nonnull character tokens taken from the basic |
c827f760 | 2863 | execution character set of Standard C@. This set consists of the ten |
72d2299c PE |
2864 | digits, the 52 lower- and upper-case English letters, and the |
2865 | characters in the following C-language string: | |
2866 | ||
2867 | @example | |
2868 | "\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~" | |
2869 | @end example | |
2870 | ||
f8e1c9e5 AD |
2871 | The @code{yylex} function and Bison must use a consistent character set |
2872 | and encoding for character tokens. For example, if you run Bison in an | |
2873 | @acronym{ASCII} environment, but then compile and run the resulting | |
2874 | program in an environment that uses an incompatible character set like | |
2875 | @acronym{EBCDIC}, the resulting program may not work because the tables | |
2876 | generated by Bison will assume @acronym{ASCII} numeric values for | |
2877 | character tokens. It is standard practice for software distributions to | |
2878 | contain C source files that were generated by Bison in an | |
2879 | @acronym{ASCII} environment, so installers on platforms that are | |
2880 | incompatible with @acronym{ASCII} must rebuild those files before | |
2881 | compiling them. | |
e966383b | 2882 | |
bfa74976 RS |
2883 | The symbol @code{error} is a terminal symbol reserved for error recovery |
2884 | (@pxref{Error Recovery}); you shouldn't use it for any other purpose. | |
23c5a174 AD |
2885 | In particular, @code{yylex} should never return this value. The default |
2886 | value of the error token is 256, unless you explicitly assigned 256 to | |
2887 | one of your tokens with a @code{%token} declaration. | |
bfa74976 | 2888 | |
342b8b6e | 2889 | @node Rules |
bfa74976 RS |
2890 | @section Syntax of Grammar Rules |
2891 | @cindex rule syntax | |
2892 | @cindex grammar rule syntax | |
2893 | @cindex syntax of grammar rules | |
2894 | ||
2895 | A Bison grammar rule has the following general form: | |
2896 | ||
2897 | @example | |
e425e872 | 2898 | @group |
bfa74976 RS |
2899 | @var{result}: @var{components}@dots{} |
2900 | ; | |
e425e872 | 2901 | @end group |
bfa74976 RS |
2902 | @end example |
2903 | ||
2904 | @noindent | |
9ecbd125 | 2905 | where @var{result} is the nonterminal symbol that this rule describes, |
bfa74976 | 2906 | and @var{components} are various terminal and nonterminal symbols that |
13863333 | 2907 | are put together by this rule (@pxref{Symbols}). |
bfa74976 RS |
2908 | |
2909 | For example, | |
2910 | ||
2911 | @example | |
2912 | @group | |
2913 | exp: exp '+' exp | |
2914 | ; | |
2915 | @end group | |
2916 | @end example | |
2917 | ||
2918 | @noindent | |
2919 | says that two groupings of type @code{exp}, with a @samp{+} token in between, | |
2920 | can be combined into a larger grouping of type @code{exp}. | |
2921 | ||
72d2299c PE |
2922 | White space in rules is significant only to separate symbols. You can add |
2923 | extra white space as you wish. | |
bfa74976 RS |
2924 | |
2925 | Scattered among the components can be @var{actions} that determine | |
2926 | the semantics of the rule. An action looks like this: | |
2927 | ||
2928 | @example | |
2929 | @{@var{C statements}@} | |
2930 | @end example | |
2931 | ||
2932 | @noindent | |
287c78f6 PE |
2933 | @cindex braced code |
2934 | This is an example of @dfn{braced code}, that is, C code surrounded by | |
2935 | braces, much like a compound statement in C@. Braced code can contain | |
2936 | any sequence of C tokens, so long as its braces are balanced. Bison | |
2937 | does not check the braced code for correctness directly; it merely | |
2938 | copies the code to the output file, where the C compiler can check it. | |
2939 | ||
2940 | Within braced code, the balanced-brace count is not affected by braces | |
2941 | within comments, string literals, or character constants, but it is | |
2942 | affected by the C digraphs @samp{<%} and @samp{%>} that represent | |
2943 | braces. At the top level braced code must be terminated by @samp{@}} | |
2944 | and not by a digraph. Bison does not look for trigraphs, so if braced | |
2945 | code uses trigraphs you should ensure that they do not affect the | |
2946 | nesting of braces or the boundaries of comments, string literals, or | |
2947 | character constants. | |
2948 | ||
bfa74976 RS |
2949 | Usually there is only one action and it follows the components. |
2950 | @xref{Actions}. | |
2951 | ||
2952 | @findex | | |
2953 | Multiple rules for the same @var{result} can be written separately or can | |
2954 | be joined with the vertical-bar character @samp{|} as follows: | |
2955 | ||
bfa74976 RS |
2956 | @example |
2957 | @group | |
2958 | @var{result}: @var{rule1-components}@dots{} | |
2959 | | @var{rule2-components}@dots{} | |
2960 | @dots{} | |
2961 | ; | |
2962 | @end group | |
2963 | @end example | |
bfa74976 RS |
2964 | |
2965 | @noindent | |
2966 | They are still considered distinct rules even when joined in this way. | |
2967 | ||
2968 | If @var{components} in a rule is empty, it means that @var{result} can | |
2969 | match the empty string. For example, here is how to define a | |
2970 | comma-separated sequence of zero or more @code{exp} groupings: | |
2971 | ||
2972 | @example | |
2973 | @group | |
2974 | expseq: /* empty */ | |
2975 | | expseq1 | |
2976 | ; | |
2977 | @end group | |
2978 | ||
2979 | @group | |
2980 | expseq1: exp | |
2981 | | expseq1 ',' exp | |
2982 | ; | |
2983 | @end group | |
2984 | @end example | |
2985 | ||
2986 | @noindent | |
2987 | It is customary to write a comment @samp{/* empty */} in each rule | |
2988 | with no components. | |
2989 | ||
342b8b6e | 2990 | @node Recursion |
bfa74976 RS |
2991 | @section Recursive Rules |
2992 | @cindex recursive rule | |
2993 | ||
f8e1c9e5 AD |
2994 | A rule is called @dfn{recursive} when its @var{result} nonterminal |
2995 | appears also on its right hand side. Nearly all Bison grammars need to | |
2996 | use recursion, because that is the only way to define a sequence of any | |
2997 | number of a particular thing. Consider this recursive definition of a | |
9ecbd125 | 2998 | comma-separated sequence of one or more expressions: |
bfa74976 RS |
2999 | |
3000 | @example | |
3001 | @group | |
3002 | expseq1: exp | |
3003 | | expseq1 ',' exp | |
3004 | ; | |
3005 | @end group | |
3006 | @end example | |
3007 | ||
3008 | @cindex left recursion | |
3009 | @cindex right recursion | |
3010 | @noindent | |
3011 | Since the recursive use of @code{expseq1} is the leftmost symbol in the | |
3012 | right hand side, we call this @dfn{left recursion}. By contrast, here | |
3013 | the same construct is defined using @dfn{right recursion}: | |
3014 | ||
3015 | @example | |
3016 | @group | |
3017 | expseq1: exp | |
3018 | | exp ',' expseq1 | |
3019 | ; | |
3020 | @end group | |
3021 | @end example | |
3022 | ||
3023 | @noindent | |
ec3bc396 AD |
3024 | Any kind of sequence can be defined using either left recursion or right |
3025 | recursion, but you should always use left recursion, because it can | |
3026 | parse a sequence of any number of elements with bounded stack space. | |
3027 | Right recursion uses up space on the Bison stack in proportion to the | |
3028 | number of elements in the sequence, because all the elements must be | |
3029 | shifted onto the stack before the rule can be applied even once. | |
3030 | @xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation | |
3031 | of this. | |
bfa74976 RS |
3032 | |
3033 | @cindex mutual recursion | |
3034 | @dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the | |
3035 | rule does not appear directly on its right hand side, but does appear | |
3036 | in rules for other nonterminals which do appear on its right hand | |
13863333 | 3037 | side. |
bfa74976 RS |
3038 | |
3039 | For example: | |
3040 | ||
3041 | @example | |
3042 | @group | |
3043 | expr: primary | |
3044 | | primary '+' primary | |
3045 | ; | |
3046 | @end group | |
3047 | ||
3048 | @group | |
3049 | primary: constant | |
3050 | | '(' expr ')' | |
3051 | ; | |
3052 | @end group | |
3053 | @end example | |
3054 | ||
3055 | @noindent | |
3056 | defines two mutually-recursive nonterminals, since each refers to the | |
3057 | other. | |
3058 | ||
342b8b6e | 3059 | @node Semantics |
bfa74976 RS |
3060 | @section Defining Language Semantics |
3061 | @cindex defining language semantics | |
13863333 | 3062 | @cindex language semantics, defining |
bfa74976 RS |
3063 | |
3064 | The grammar rules for a language determine only the syntax. The semantics | |
3065 | are determined by the semantic values associated with various tokens and | |
3066 | groupings, and by the actions taken when various groupings are recognized. | |
3067 | ||
3068 | For example, the calculator calculates properly because the value | |
3069 | associated with each expression is the proper number; it adds properly | |
3070 | because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add | |
3071 | the numbers associated with @var{x} and @var{y}. | |
3072 | ||
3073 | @menu | |
3074 | * Value Type:: Specifying one data type for all semantic values. | |
3075 | * Multiple Types:: Specifying several alternative data types. | |
3076 | * Actions:: An action is the semantic definition of a grammar rule. | |
3077 | * Action Types:: Specifying data types for actions to operate on. | |
3078 | * Mid-Rule Actions:: Most actions go at the end of a rule. | |
3079 | This says when, why and how to use the exceptional | |
3080 | action in the middle of a rule. | |
3081 | @end menu | |
3082 | ||
342b8b6e | 3083 | @node Value Type |
bfa74976 RS |
3084 | @subsection Data Types of Semantic Values |
3085 | @cindex semantic value type | |
3086 | @cindex value type, semantic | |
3087 | @cindex data types of semantic values | |
3088 | @cindex default data type | |
3089 | ||
3090 | In a simple program it may be sufficient to use the same data type for | |
3091 | the semantic values of all language constructs. This was true in the | |
c827f760 | 3092 | @acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish |
1964ad8c | 3093 | Notation Calculator}). |
bfa74976 RS |
3094 | |
3095 | Bison's default is to use type @code{int} for all semantic values. To | |
3096 | specify some other type, define @code{YYSTYPE} as a macro, like this: | |
3097 | ||
3098 | @example | |
3099 | #define YYSTYPE double | |
3100 | @end example | |
3101 | ||
3102 | @noindent | |
50cce58e PE |
3103 | @code{YYSTYPE}'s replacement list should be a type name |
3104 | that does not contain parentheses or square brackets. | |
342b8b6e | 3105 | This macro definition must go in the prologue of the grammar file |
75f5aaea | 3106 | (@pxref{Grammar Outline, ,Outline of a Bison Grammar}). |
bfa74976 | 3107 | |
342b8b6e | 3108 | @node Multiple Types |
bfa74976 RS |
3109 | @subsection More Than One Value Type |
3110 | ||
3111 | In most programs, you will need different data types for different kinds | |
3112 | of tokens and groupings. For example, a numeric constant may need type | |
f8e1c9e5 AD |
3113 | @code{int} or @code{long int}, while a string constant needs type |
3114 | @code{char *}, and an identifier might need a pointer to an entry in the | |
3115 | symbol table. | |
bfa74976 RS |
3116 | |
3117 | To use more than one data type for semantic values in one parser, Bison | |
3118 | requires you to do two things: | |
3119 | ||
3120 | @itemize @bullet | |
3121 | @item | |
3122 | Specify the entire collection of possible data types, with the | |
704a47c4 AD |
3123 | @code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of |
3124 | Value Types}). | |
bfa74976 RS |
3125 | |
3126 | @item | |
14ded682 AD |
3127 | Choose one of those types for each symbol (terminal or nonterminal) for |
3128 | which semantic values are used. This is done for tokens with the | |
3129 | @code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names}) | |
3130 | and for groupings with the @code{%type} Bison declaration (@pxref{Type | |
3131 | Decl, ,Nonterminal Symbols}). | |
bfa74976 RS |
3132 | @end itemize |
3133 | ||
342b8b6e | 3134 | @node Actions |
bfa74976 RS |
3135 | @subsection Actions |
3136 | @cindex action | |
3137 | @vindex $$ | |
3138 | @vindex $@var{n} | |
3139 | ||
3140 | An action accompanies a syntactic rule and contains C code to be executed | |
3141 | each time an instance of that rule is recognized. The task of most actions | |
3142 | is to compute a semantic value for the grouping built by the rule from the | |
3143 | semantic values associated with tokens or smaller groupings. | |
3144 | ||
287c78f6 PE |
3145 | An action consists of braced code containing C statements, and can be |
3146 | placed at any position in the rule; | |
704a47c4 AD |
3147 | it is executed at that position. Most rules have just one action at the |
3148 | end of the rule, following all the components. Actions in the middle of | |
3149 | a rule are tricky and used only for special purposes (@pxref{Mid-Rule | |
3150 | Actions, ,Actions in Mid-Rule}). | |
bfa74976 RS |
3151 | |
3152 | The C code in an action can refer to the semantic values of the components | |
3153 | matched by the rule with the construct @code{$@var{n}}, which stands for | |
3154 | the value of the @var{n}th component. The semantic value for the grouping | |
0cc3da3a PE |
3155 | being constructed is @code{$$}. Bison translates both of these |
3156 | constructs into expressions of the appropriate type when it copies the | |
3157 | actions into the parser file. @code{$$} is translated to a modifiable | |
3158 | lvalue, so it can be assigned to. | |
bfa74976 RS |
3159 | |
3160 | Here is a typical example: | |
3161 | ||
3162 | @example | |
3163 | @group | |
3164 | exp: @dots{} | |
3165 | | exp '+' exp | |
3166 | @{ $$ = $1 + $3; @} | |
3167 | @end group | |
3168 | @end example | |
3169 | ||
3170 | @noindent | |
3171 | This rule constructs an @code{exp} from two smaller @code{exp} groupings | |
3172 | connected by a plus-sign token. In the action, @code{$1} and @code{$3} | |
3173 | refer to the semantic values of the two component @code{exp} groupings, | |
3174 | which are the first and third symbols on the right hand side of the rule. | |
3175 | The sum is stored into @code{$$} so that it becomes the semantic value of | |
3176 | the addition-expression just recognized by the rule. If there were a | |
3177 | useful semantic value associated with the @samp{+} token, it could be | |
e0c471a9 | 3178 | referred to as @code{$2}. |
bfa74976 | 3179 | |
3ded9a63 AD |
3180 | Note that the vertical-bar character @samp{|} is really a rule |
3181 | separator, and actions are attached to a single rule. This is a | |
3182 | difference with tools like Flex, for which @samp{|} stands for either | |
3183 | ``or'', or ``the same action as that of the next rule''. In the | |
3184 | following example, the action is triggered only when @samp{b} is found: | |
3185 | ||
3186 | @example | |
3187 | @group | |
3188 | a-or-b: 'a'|'b' @{ a_or_b_found = 1; @}; | |
3189 | @end group | |
3190 | @end example | |
3191 | ||
bfa74976 RS |
3192 | @cindex default action |
3193 | If you don't specify an action for a rule, Bison supplies a default: | |
72f889cc AD |
3194 | @w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule |
3195 | becomes the value of the whole rule. Of course, the default action is | |
3196 | valid only if the two data types match. There is no meaningful default | |
3197 | action for an empty rule; every empty rule must have an explicit action | |
3198 | unless the rule's value does not matter. | |
bfa74976 RS |
3199 | |
3200 | @code{$@var{n}} with @var{n} zero or negative is allowed for reference | |
3201 | to tokens and groupings on the stack @emph{before} those that match the | |
3202 | current rule. This is a very risky practice, and to use it reliably | |
3203 | you must be certain of the context in which the rule is applied. Here | |
3204 | is a case in which you can use this reliably: | |
3205 | ||
3206 | @example | |
3207 | @group | |
3208 | foo: expr bar '+' expr @{ @dots{} @} | |
3209 | | expr bar '-' expr @{ @dots{} @} | |
3210 | ; | |
3211 | @end group | |
3212 | ||
3213 | @group | |
3214 | bar: /* empty */ | |
3215 | @{ previous_expr = $0; @} | |
3216 | ; | |
3217 | @end group | |
3218 | @end example | |
3219 | ||
3220 | As long as @code{bar} is used only in the fashion shown here, @code{$0} | |
3221 | always refers to the @code{expr} which precedes @code{bar} in the | |
3222 | definition of @code{foo}. | |
3223 | ||
32c29292 | 3224 | @vindex yylval |
742e4900 | 3225 | It is also possible to access the semantic value of the lookahead token, if |
32c29292 JD |
3226 | any, from a semantic action. |
3227 | This semantic value is stored in @code{yylval}. | |
3228 | @xref{Action Features, ,Special Features for Use in Actions}. | |
3229 | ||
342b8b6e | 3230 | @node Action Types |
bfa74976 RS |
3231 | @subsection Data Types of Values in Actions |
3232 | @cindex action data types | |
3233 | @cindex data types in actions | |
3234 | ||
3235 | If you have chosen a single data type for semantic values, the @code{$$} | |
3236 | and @code{$@var{n}} constructs always have that data type. | |
3237 | ||
3238 | If you have used @code{%union} to specify a variety of data types, then you | |
3239 | must declare a choice among these types for each terminal or nonterminal | |
3240 | symbol that can have a semantic value. Then each time you use @code{$$} or | |
3241 | @code{$@var{n}}, its data type is determined by which symbol it refers to | |
e0c471a9 | 3242 | in the rule. In this example, |
bfa74976 RS |
3243 | |
3244 | @example | |
3245 | @group | |
3246 | exp: @dots{} | |
3247 | | exp '+' exp | |
3248 | @{ $$ = $1 + $3; @} | |
3249 | @end group | |
3250 | @end example | |
3251 | ||
3252 | @noindent | |
3253 | @code{$1} and @code{$3} refer to instances of @code{exp}, so they all | |
3254 | have the data type declared for the nonterminal symbol @code{exp}. If | |
3255 | @code{$2} were used, it would have the data type declared for the | |
e0c471a9 | 3256 | terminal symbol @code{'+'}, whatever that might be. |
bfa74976 RS |
3257 | |
3258 | Alternatively, you can specify the data type when you refer to the value, | |
3259 | by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the | |
3260 | reference. For example, if you have defined types as shown here: | |
3261 | ||
3262 | @example | |
3263 | @group | |
3264 | %union @{ | |
3265 | int itype; | |
3266 | double dtype; | |
3267 | @} | |
3268 | @end group | |
3269 | @end example | |
3270 | ||
3271 | @noindent | |
3272 | then you can write @code{$<itype>1} to refer to the first subunit of the | |
3273 | rule as an integer, or @code{$<dtype>1} to refer to it as a double. | |
3274 | ||
342b8b6e | 3275 | @node Mid-Rule Actions |
bfa74976 RS |
3276 | @subsection Actions in Mid-Rule |
3277 | @cindex actions in mid-rule | |
3278 | @cindex mid-rule actions | |
3279 | ||
3280 | Occasionally it is useful to put an action in the middle of a rule. | |
3281 | These actions are written just like usual end-of-rule actions, but they | |
3282 | are executed before the parser even recognizes the following components. | |
3283 | ||
3284 | A mid-rule action may refer to the components preceding it using | |
3285 | @code{$@var{n}}, but it may not refer to subsequent components because | |
3286 | it is run before they are parsed. | |
3287 | ||
3288 | The mid-rule action itself counts as one of the components of the rule. | |
3289 | This makes a difference when there is another action later in the same rule | |
3290 | (and usually there is another at the end): you have to count the actions | |
3291 | along with the symbols when working out which number @var{n} to use in | |
3292 | @code{$@var{n}}. | |
3293 | ||
3294 | The mid-rule action can also have a semantic value. The action can set | |
3295 | its value with an assignment to @code{$$}, and actions later in the rule | |
3296 | can refer to the value using @code{$@var{n}}. Since there is no symbol | |
3297 | to name the action, there is no way to declare a data type for the value | |
fdc6758b MA |
3298 | in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to |
3299 | specify a data type each time you refer to this value. | |
bfa74976 RS |
3300 | |
3301 | There is no way to set the value of the entire rule with a mid-rule | |
3302 | action, because assignments to @code{$$} do not have that effect. The | |
3303 | only way to set the value for the entire rule is with an ordinary action | |
3304 | at the end of the rule. | |
3305 | ||
3306 | Here is an example from a hypothetical compiler, handling a @code{let} | |
3307 | statement that looks like @samp{let (@var{variable}) @var{statement}} and | |
3308 | serves to create a variable named @var{variable} temporarily for the | |
3309 | duration of @var{statement}. To parse this construct, we must put | |
3310 | @var{variable} into the symbol table while @var{statement} is parsed, then | |
3311 | remove it afterward. Here is how it is done: | |
3312 | ||
3313 | @example | |
3314 | @group | |
3315 | stmt: LET '(' var ')' | |
3316 | @{ $<context>$ = push_context (); | |
3317 | declare_variable ($3); @} | |
3318 | stmt @{ $$ = $6; | |
3319 | pop_context ($<context>5); @} | |
3320 | @end group | |
3321 | @end example | |
3322 | ||
3323 | @noindent | |
3324 | As soon as @samp{let (@var{variable})} has been recognized, the first | |
3325 | action is run. It saves a copy of the current semantic context (the | |
3326 | list of accessible variables) as its semantic value, using alternative | |
3327 | @code{context} in the data-type union. Then it calls | |
3328 | @code{declare_variable} to add the new variable to that list. Once the | |
3329 | first action is finished, the embedded statement @code{stmt} can be | |
3330 | parsed. Note that the mid-rule action is component number 5, so the | |
3331 | @samp{stmt} is component number 6. | |
3332 | ||
3333 | After the embedded statement is parsed, its semantic value becomes the | |
3334 | value of the entire @code{let}-statement. Then the semantic value from the | |
3335 | earlier action is used to restore the prior list of variables. This | |
3336 | removes the temporary @code{let}-variable from the list so that it won't | |
3337 | appear to exist while the rest of the program is parsed. | |
3338 | ||
841a7737 JD |
3339 | @findex %destructor |
3340 | @cindex discarded symbols, mid-rule actions | |
3341 | @cindex error recovery, mid-rule actions | |
3342 | In the above example, if the parser initiates error recovery (@pxref{Error | |
3343 | Recovery}) while parsing the tokens in the embedded statement @code{stmt}, | |
3344 | it might discard the previous semantic context @code{$<context>5} without | |
3345 | restoring it. | |
3346 | Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing | |
3347 | Discarded Symbols}). | |
3348 | However, Bison currently provides no means to declare a destructor for a | |
3349 | mid-rule action's semantic value. | |
3350 | ||
3351 | One solution is to bury the mid-rule action inside a nonterminal symbol and to | |
3352 | declare a destructor for that symbol: | |
3353 | ||
3354 | @example | |
3355 | @group | |
3356 | %type <context> let | |
3357 | %destructor @{ pop_context ($$); @} let | |
3358 | ||
3359 | %% | |
3360 | ||
3361 | stmt: let stmt | |
3362 | @{ $$ = $2; | |
3363 | pop_context ($1); @} | |
3364 | ; | |
3365 | ||
3366 | let: LET '(' var ')' | |
3367 | @{ $$ = push_context (); | |
3368 | declare_variable ($3); @} | |
3369 | ; | |
3370 | ||
3371 | @end group | |
3372 | @end example | |
3373 | ||
3374 | @noindent | |
3375 | Note that the action is now at the end of its rule. | |
3376 | Any mid-rule action can be converted to an end-of-rule action in this way, and | |
3377 | this is what Bison actually does to implement mid-rule actions. | |
3378 | ||
bfa74976 RS |
3379 | Taking action before a rule is completely recognized often leads to |
3380 | conflicts since the parser must commit to a parse in order to execute the | |
3381 | action. For example, the following two rules, without mid-rule actions, | |
3382 | can coexist in a working parser because the parser can shift the open-brace | |
3383 | token and look at what follows before deciding whether there is a | |
3384 | declaration or not: | |
3385 | ||
3386 | @example | |
3387 | @group | |
3388 | compound: '@{' declarations statements '@}' | |
3389 | | '@{' statements '@}' | |
3390 | ; | |
3391 | @end group | |
3392 | @end example | |
3393 | ||
3394 | @noindent | |
3395 | But when we add a mid-rule action as follows, the rules become nonfunctional: | |
3396 | ||
3397 | @example | |
3398 | @group | |
3399 | compound: @{ prepare_for_local_variables (); @} | |
3400 | '@{' declarations statements '@}' | |
3401 | @end group | |
3402 | @group | |
3403 | | '@{' statements '@}' | |
3404 | ; | |
3405 | @end group | |
3406 | @end example | |
3407 | ||
3408 | @noindent | |
3409 | Now the parser is forced to decide whether to run the mid-rule action | |
3410 | when it has read no farther than the open-brace. In other words, it | |
3411 | must commit to using one rule or the other, without sufficient | |
3412 | information to do it correctly. (The open-brace token is what is called | |
742e4900 JD |
3413 | the @dfn{lookahead} token at this time, since the parser is still |
3414 | deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.) | |
bfa74976 RS |
3415 | |
3416 | You might think that you could correct the problem by putting identical | |
3417 | actions into the two rules, like this: | |
3418 | ||
3419 | @example | |
3420 | @group | |
3421 | compound: @{ prepare_for_local_variables (); @} | |
3422 | '@{' declarations statements '@}' | |
3423 | | @{ prepare_for_local_variables (); @} | |
3424 | '@{' statements '@}' | |
3425 | ; | |
3426 | @end group | |
3427 | @end example | |
3428 | ||
3429 | @noindent | |
3430 | But this does not help, because Bison does not realize that the two actions | |
3431 | are identical. (Bison never tries to understand the C code in an action.) | |
3432 | ||
3433 | If the grammar is such that a declaration can be distinguished from a | |
3434 | statement by the first token (which is true in C), then one solution which | |
3435 | does work is to put the action after the open-brace, like this: | |
3436 | ||
3437 | @example | |
3438 | @group | |
3439 | compound: '@{' @{ prepare_for_local_variables (); @} | |
3440 | declarations statements '@}' | |
3441 | | '@{' statements '@}' | |
3442 | ; | |
3443 | @end group | |
3444 | @end example | |
3445 | ||
3446 | @noindent | |
3447 | Now the first token of the following declaration or statement, | |
3448 | which would in any case tell Bison which rule to use, can still do so. | |
3449 | ||
3450 | Another solution is to bury the action inside a nonterminal symbol which | |
3451 | serves as a subroutine: | |
3452 | ||
3453 | @example | |
3454 | @group | |
3455 | subroutine: /* empty */ | |
3456 | @{ prepare_for_local_variables (); @} | |
3457 | ; | |
3458 | ||
3459 | @end group | |
3460 | ||
3461 | @group | |
3462 | compound: subroutine | |
3463 | '@{' declarations statements '@}' | |
3464 | | subroutine | |
3465 | '@{' statements '@}' | |
3466 | ; | |
3467 | @end group | |
3468 | @end example | |
3469 | ||
3470 | @noindent | |
3471 | Now Bison can execute the action in the rule for @code{subroutine} without | |
841a7737 | 3472 | deciding which rule for @code{compound} it will eventually use. |
bfa74976 | 3473 | |
342b8b6e | 3474 | @node Locations |
847bf1f5 AD |
3475 | @section Tracking Locations |
3476 | @cindex location | |
95923bd6 AD |
3477 | @cindex textual location |
3478 | @cindex location, textual | |
847bf1f5 AD |
3479 | |
3480 | Though grammar rules and semantic actions are enough to write a fully | |
72d2299c | 3481 | functional parser, it can be useful to process some additional information, |
3e259915 MA |
3482 | especially symbol locations. |
3483 | ||
704a47c4 AD |
3484 | The way locations are handled is defined by providing a data type, and |
3485 | actions to take when rules are matched. | |
847bf1f5 AD |
3486 | |
3487 | @menu | |
3488 | * Location Type:: Specifying a data type for locations. | |
3489 | * Actions and Locations:: Using locations in actions. | |
3490 | * Location Default Action:: Defining a general way to compute locations. | |
3491 | @end menu | |
3492 | ||
342b8b6e | 3493 | @node Location Type |
847bf1f5 AD |
3494 | @subsection Data Type of Locations |
3495 | @cindex data type of locations | |
3496 | @cindex default location type | |
3497 | ||
3498 | Defining a data type for locations is much simpler than for semantic values, | |
3499 | since all tokens and groupings always use the same type. | |
3500 | ||
50cce58e PE |
3501 | You can specify the type of locations by defining a macro called |
3502 | @code{YYLTYPE}, just as you can specify the semantic value type by | |
3503 | defining @code{YYSTYPE} (@pxref{Value Type}). | |
847bf1f5 AD |
3504 | When @code{YYLTYPE} is not defined, Bison uses a default structure type with |
3505 | four members: | |
3506 | ||
3507 | @example | |
6273355b | 3508 | typedef struct YYLTYPE |
847bf1f5 AD |
3509 | @{ |
3510 | int first_line; | |
3511 | int first_column; | |
3512 | int last_line; | |
3513 | int last_column; | |
6273355b | 3514 | @} YYLTYPE; |
847bf1f5 AD |
3515 | @end example |
3516 | ||
cd48d21d AD |
3517 | At the beginning of the parsing, Bison initializes all these fields to 1 |
3518 | for @code{yylloc}. | |
3519 | ||
342b8b6e | 3520 | @node Actions and Locations |
847bf1f5 AD |
3521 | @subsection Actions and Locations |
3522 | @cindex location actions | |
3523 | @cindex actions, location | |
3524 | @vindex @@$ | |
3525 | @vindex @@@var{n} | |
3526 | ||
3527 | Actions are not only useful for defining language semantics, but also for | |
3528 | describing the behavior of the output parser with locations. | |
3529 | ||
3530 | The most obvious way for building locations of syntactic groupings is very | |
72d2299c | 3531 | similar to the way semantic values are computed. In a given rule, several |
847bf1f5 AD |
3532 | constructs can be used to access the locations of the elements being matched. |
3533 | The location of the @var{n}th component of the right hand side is | |
3534 | @code{@@@var{n}}, while the location of the left hand side grouping is | |
3535 | @code{@@$}. | |
3536 | ||
3e259915 | 3537 | Here is a basic example using the default data type for locations: |
847bf1f5 AD |
3538 | |
3539 | @example | |
3540 | @group | |
3541 | exp: @dots{} | |
3e259915 | 3542 | | exp '/' exp |
847bf1f5 | 3543 | @{ |
3e259915 MA |
3544 | @@$.first_column = @@1.first_column; |
3545 | @@$.first_line = @@1.first_line; | |
847bf1f5 AD |
3546 | @@$.last_column = @@3.last_column; |
3547 | @@$.last_line = @@3.last_line; | |
3e259915 MA |
3548 | if ($3) |
3549 | $$ = $1 / $3; | |
3550 | else | |
3551 | @{ | |
3552 | $$ = 1; | |
4e03e201 AD |
3553 | fprintf (stderr, |
3554 | "Division by zero, l%d,c%d-l%d,c%d", | |
3555 | @@3.first_line, @@3.first_column, | |
3556 | @@3.last_line, @@3.last_column); | |
3e259915 | 3557 | @} |
847bf1f5 AD |
3558 | @} |
3559 | @end group | |
3560 | @end example | |
3561 | ||
3e259915 | 3562 | As for semantic values, there is a default action for locations that is |
72d2299c | 3563 | run each time a rule is matched. It sets the beginning of @code{@@$} to the |
3e259915 | 3564 | beginning of the first symbol, and the end of @code{@@$} to the end of the |
79282c6c | 3565 | last symbol. |
3e259915 | 3566 | |
72d2299c | 3567 | With this default action, the location tracking can be fully automatic. The |
3e259915 MA |
3568 | example above simply rewrites this way: |
3569 | ||
3570 | @example | |
3571 | @group | |
3572 | exp: @dots{} | |
3573 | | exp '/' exp | |
3574 | @{ | |
3575 | if ($3) | |
3576 | $$ = $1 / $3; | |
3577 | else | |
3578 | @{ | |
3579 | $$ = 1; | |
4e03e201 AD |
3580 | fprintf (stderr, |
3581 | "Division by zero, l%d,c%d-l%d,c%d", | |
3582 | @@3.first_line, @@3.first_column, | |
3583 | @@3.last_line, @@3.last_column); | |
3e259915 MA |
3584 | @} |
3585 | @} | |
3586 | @end group | |
3587 | @end example | |
847bf1f5 | 3588 | |
32c29292 | 3589 | @vindex yylloc |
742e4900 | 3590 | It is also possible to access the location of the lookahead token, if any, |
32c29292 JD |
3591 | from a semantic action. |
3592 | This location is stored in @code{yylloc}. | |
3593 | @xref{Action Features, ,Special Features for Use in Actions}. | |
3594 | ||
342b8b6e | 3595 | @node Location Default Action |
847bf1f5 AD |
3596 | @subsection Default Action for Locations |
3597 | @vindex YYLLOC_DEFAULT | |
8710fc41 | 3598 | @cindex @acronym{GLR} parsers and @code{YYLLOC_DEFAULT} |
847bf1f5 | 3599 | |
72d2299c | 3600 | Actually, actions are not the best place to compute locations. Since |
704a47c4 AD |
3601 | locations are much more general than semantic values, there is room in |
3602 | the output parser to redefine the default action to take for each | |
72d2299c | 3603 | rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is |
96b93a3d PE |
3604 | matched, before the associated action is run. It is also invoked |
3605 | while processing a syntax error, to compute the error's location. | |
8710fc41 JD |
3606 | Before reporting an unresolvable syntactic ambiguity, a @acronym{GLR} |
3607 | parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location | |
3608 | of that ambiguity. | |
847bf1f5 | 3609 | |
3e259915 | 3610 | Most of the time, this macro is general enough to suppress location |
79282c6c | 3611 | dedicated code from semantic actions. |
847bf1f5 | 3612 | |
72d2299c | 3613 | The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is |
96b93a3d | 3614 | the location of the grouping (the result of the computation). When a |
766de5eb | 3615 | rule is matched, the second parameter identifies locations of |
96b93a3d | 3616 | all right hand side elements of the rule being matched, and the third |
8710fc41 JD |
3617 | parameter is the size of the rule's right hand side. |
3618 | When a @acronym{GLR} parser reports an ambiguity, which of multiple candidate | |
3619 | right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined. | |
3620 | When processing a syntax error, the second parameter identifies locations | |
3621 | of the symbols that were discarded during error processing, and the third | |
96b93a3d | 3622 | parameter is the number of discarded symbols. |
847bf1f5 | 3623 | |
766de5eb | 3624 | By default, @code{YYLLOC_DEFAULT} is defined this way: |
847bf1f5 | 3625 | |
766de5eb | 3626 | @smallexample |
847bf1f5 | 3627 | @group |
766de5eb PE |
3628 | # define YYLLOC_DEFAULT(Current, Rhs, N) \ |
3629 | do \ | |
3630 | if (N) \ | |
3631 | @{ \ | |
3632 | (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \ | |
3633 | (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \ | |
3634 | (Current).last_line = YYRHSLOC(Rhs, N).last_line; \ | |
3635 | (Current).last_column = YYRHSLOC(Rhs, N).last_column; \ | |
3636 | @} \ | |
3637 | else \ | |
3638 | @{ \ | |
3639 | (Current).first_line = (Current).last_line = \ | |
3640 | YYRHSLOC(Rhs, 0).last_line; \ | |
3641 | (Current).first_column = (Current).last_column = \ | |
3642 | YYRHSLOC(Rhs, 0).last_column; \ | |
3643 | @} \ | |
3644 | while (0) | |
847bf1f5 | 3645 | @end group |
766de5eb | 3646 | @end smallexample |
676385e2 | 3647 | |
766de5eb PE |
3648 | where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol |
3649 | in @var{rhs} when @var{k} is positive, and the location of the symbol | |
f28ac696 | 3650 | just before the reduction when @var{k} and @var{n} are both zero. |
676385e2 | 3651 | |
3e259915 | 3652 | When defining @code{YYLLOC_DEFAULT}, you should consider that: |
847bf1f5 | 3653 | |
3e259915 | 3654 | @itemize @bullet |
79282c6c | 3655 | @item |
72d2299c | 3656 | All arguments are free of side-effects. However, only the first one (the |
3e259915 | 3657 | result) should be modified by @code{YYLLOC_DEFAULT}. |
847bf1f5 | 3658 | |
3e259915 | 3659 | @item |
766de5eb PE |
3660 | For consistency with semantic actions, valid indexes within the |
3661 | right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a | |
3662 | valid index, and it refers to the symbol just before the reduction. | |
3663 | During error processing @var{n} is always positive. | |
0ae99356 PE |
3664 | |
3665 | @item | |
3666 | Your macro should parenthesize its arguments, if need be, since the | |
3667 | actual arguments may not be surrounded by parentheses. Also, your | |
3668 | macro should expand to something that can be used as a single | |
3669 | statement when it is followed by a semicolon. | |
3e259915 | 3670 | @end itemize |
847bf1f5 | 3671 | |
342b8b6e | 3672 | @node Declarations |
bfa74976 RS |
3673 | @section Bison Declarations |
3674 | @cindex declarations, Bison | |
3675 | @cindex Bison declarations | |
3676 | ||
3677 | The @dfn{Bison declarations} section of a Bison grammar defines the symbols | |
3678 | used in formulating the grammar and the data types of semantic values. | |
3679 | @xref{Symbols}. | |
3680 | ||
3681 | All token type names (but not single-character literal tokens such as | |
3682 | @code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be | |
3683 | declared if you need to specify which data type to use for the semantic | |
3684 | value (@pxref{Multiple Types, ,More Than One Value Type}). | |
3685 | ||
3686 | The first rule in the file also specifies the start symbol, by default. | |
3687 | If you want some other symbol to be the start symbol, you must declare | |
704a47c4 AD |
3688 | it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free |
3689 | Grammars}). | |
bfa74976 RS |
3690 | |
3691 | @menu | |
b50d2359 | 3692 | * Require Decl:: Requiring a Bison version. |
bfa74976 RS |
3693 | * Token Decl:: Declaring terminal symbols. |
3694 | * Precedence Decl:: Declaring terminals with precedence and associativity. | |
3695 | * Union Decl:: Declaring the set of all semantic value types. | |
3696 | * Type Decl:: Declaring the choice of type for a nonterminal symbol. | |
18d192f0 | 3697 | * Initial Action Decl:: Code run before parsing starts. |
72f889cc | 3698 | * Destructor Decl:: Declaring how symbols are freed. |
d6328241 | 3699 | * Expect Decl:: Suppressing warnings about parsing conflicts. |
bfa74976 RS |
3700 | * Start Decl:: Specifying the start symbol. |
3701 | * Pure Decl:: Requesting a reentrant parser. | |
3702 | * Decl Summary:: Table of all Bison declarations. | |
3703 | @end menu | |
3704 | ||
b50d2359 AD |
3705 | @node Require Decl |
3706 | @subsection Require a Version of Bison | |
3707 | @cindex version requirement | |
3708 | @cindex requiring a version of Bison | |
3709 | @findex %require | |
3710 | ||
3711 | You may require the minimum version of Bison to process the grammar. If | |
9b8a5ce0 AD |
3712 | the requirement is not met, @command{bison} exits with an error (exit |
3713 | status 63). | |
b50d2359 AD |
3714 | |
3715 | @example | |
3716 | %require "@var{version}" | |
3717 | @end example | |
3718 | ||
342b8b6e | 3719 | @node Token Decl |
bfa74976 RS |
3720 | @subsection Token Type Names |
3721 | @cindex declaring token type names | |
3722 | @cindex token type names, declaring | |
931c7513 | 3723 | @cindex declaring literal string tokens |
bfa74976 RS |
3724 | @findex %token |
3725 | ||
3726 | The basic way to declare a token type name (terminal symbol) is as follows: | |
3727 | ||
3728 | @example | |
3729 | %token @var{name} | |
3730 | @end example | |
3731 | ||
3732 | Bison will convert this into a @code{#define} directive in | |
3733 | the parser, so that the function @code{yylex} (if it is in this file) | |
3734 | can use the name @var{name} to stand for this token type's code. | |
3735 | ||
14ded682 AD |
3736 | Alternatively, you can use @code{%left}, @code{%right}, or |
3737 | @code{%nonassoc} instead of @code{%token}, if you wish to specify | |
3738 | associativity and precedence. @xref{Precedence Decl, ,Operator | |
3739 | Precedence}. | |
bfa74976 RS |
3740 | |
3741 | You can explicitly specify the numeric code for a token type by appending | |
1452af69 PE |
3742 | a decimal or hexadecimal integer value in the field immediately |
3743 | following the token name: | |
bfa74976 RS |
3744 | |
3745 | @example | |
3746 | %token NUM 300 | |
1452af69 | 3747 | %token XNUM 0x12d // a GNU extension |
bfa74976 RS |
3748 | @end example |
3749 | ||
3750 | @noindent | |
3751 | It is generally best, however, to let Bison choose the numeric codes for | |
3752 | all token types. Bison will automatically select codes that don't conflict | |
e966383b | 3753 | with each other or with normal characters. |
bfa74976 RS |
3754 | |
3755 | In the event that the stack type is a union, you must augment the | |
3756 | @code{%token} or other token declaration to include the data type | |
704a47c4 AD |
3757 | alternative delimited by angle-brackets (@pxref{Multiple Types, ,More |
3758 | Than One Value Type}). | |
bfa74976 RS |
3759 | |
3760 | For example: | |
3761 | ||
3762 | @example | |
3763 | @group | |
3764 | %union @{ /* define stack type */ | |
3765 | double val; | |
3766 | symrec *tptr; | |
3767 | @} | |
3768 | %token <val> NUM /* define token NUM and its type */ | |
3769 | @end group | |
3770 | @end example | |
3771 | ||
931c7513 RS |
3772 | You can associate a literal string token with a token type name by |
3773 | writing the literal string at the end of a @code{%token} | |
3774 | declaration which declares the name. For example: | |
3775 | ||
3776 | @example | |
3777 | %token arrow "=>" | |
3778 | @end example | |
3779 | ||
3780 | @noindent | |
3781 | For example, a grammar for the C language might specify these names with | |
3782 | equivalent literal string tokens: | |
3783 | ||
3784 | @example | |
3785 | %token <operator> OR "||" | |
3786 | %token <operator> LE 134 "<=" | |
3787 | %left OR "<=" | |
3788 | @end example | |
3789 | ||
3790 | @noindent | |
3791 | Once you equate the literal string and the token name, you can use them | |
3792 | interchangeably in further declarations or the grammar rules. The | |
3793 | @code{yylex} function can use the token name or the literal string to | |
3794 | obtain the token type code number (@pxref{Calling Convention}). | |
3795 | ||
342b8b6e | 3796 | @node Precedence Decl |
bfa74976 RS |
3797 | @subsection Operator Precedence |
3798 | @cindex precedence declarations | |
3799 | @cindex declaring operator precedence | |
3800 | @cindex operator precedence, declaring | |
3801 | ||
3802 | Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to | |
3803 | declare a token and specify its precedence and associativity, all at | |
3804 | once. These are called @dfn{precedence declarations}. | |
704a47c4 AD |
3805 | @xref{Precedence, ,Operator Precedence}, for general information on |
3806 | operator precedence. | |
bfa74976 RS |
3807 | |
3808 | The syntax of a precedence declaration is the same as that of | |
3809 | @code{%token}: either | |
3810 | ||
3811 | @example | |
3812 | %left @var{symbols}@dots{} | |
3813 | @end example | |
3814 | ||
3815 | @noindent | |
3816 | or | |
3817 | ||
3818 | @example | |
3819 | %left <@var{type}> @var{symbols}@dots{} | |
3820 | @end example | |
3821 | ||
3822 | And indeed any of these declarations serves the purposes of @code{%token}. | |
3823 | But in addition, they specify the associativity and relative precedence for | |
3824 | all the @var{symbols}: | |
3825 | ||
3826 | @itemize @bullet | |
3827 | @item | |
3828 | The associativity of an operator @var{op} determines how repeated uses | |
3829 | of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op} | |
3830 | @var{z}} is parsed by grouping @var{x} with @var{y} first or by | |
3831 | grouping @var{y} with @var{z} first. @code{%left} specifies | |
3832 | left-associativity (grouping @var{x} with @var{y} first) and | |
3833 | @code{%right} specifies right-associativity (grouping @var{y} with | |
3834 | @var{z} first). @code{%nonassoc} specifies no associativity, which | |
3835 | means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is | |
3836 | considered a syntax error. | |
3837 | ||
3838 | @item | |
3839 | The precedence of an operator determines how it nests with other operators. | |
3840 | All the tokens declared in a single precedence declaration have equal | |
3841 | precedence and nest together according to their associativity. | |
3842 | When two tokens declared in different precedence declarations associate, | |
3843 | the one declared later has the higher precedence and is grouped first. | |
3844 | @end itemize | |
3845 | ||
342b8b6e | 3846 | @node Union Decl |
bfa74976 RS |
3847 | @subsection The Collection of Value Types |
3848 | @cindex declaring value types | |
3849 | @cindex value types, declaring | |
3850 | @findex %union | |
3851 | ||
287c78f6 PE |
3852 | The @code{%union} declaration specifies the entire collection of |
3853 | possible data types for semantic values. The keyword @code{%union} is | |
3854 | followed by braced code containing the same thing that goes inside a | |
3855 | @code{union} in C@. | |
bfa74976 RS |
3856 | |
3857 | For example: | |
3858 | ||
3859 | @example | |
3860 | @group | |
3861 | %union @{ | |
3862 | double val; | |
3863 | symrec *tptr; | |
3864 | @} | |
3865 | @end group | |
3866 | @end example | |
3867 | ||
3868 | @noindent | |
3869 | This says that the two alternative types are @code{double} and @code{symrec | |
3870 | *}. They are given names @code{val} and @code{tptr}; these names are used | |
3871 | in the @code{%token} and @code{%type} declarations to pick one of the types | |
3872 | for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}). | |
3873 | ||
6273355b PE |
3874 | As an extension to @acronym{POSIX}, a tag is allowed after the |
3875 | @code{union}. For example: | |
3876 | ||
3877 | @example | |
3878 | @group | |
3879 | %union value @{ | |
3880 | double val; | |
3881 | symrec *tptr; | |
3882 | @} | |
3883 | @end group | |
3884 | @end example | |
3885 | ||
d6ca7905 | 3886 | @noindent |
6273355b PE |
3887 | specifies the union tag @code{value}, so the corresponding C type is |
3888 | @code{union value}. If you do not specify a tag, it defaults to | |
3889 | @code{YYSTYPE}. | |
3890 | ||
d6ca7905 PE |
3891 | As another extension to @acronym{POSIX}, you may specify multiple |
3892 | @code{%union} declarations; their contents are concatenated. However, | |
3893 | only the first @code{%union} declaration can specify a tag. | |
3894 | ||
6273355b | 3895 | Note that, unlike making a @code{union} declaration in C, you need not write |
bfa74976 RS |
3896 | a semicolon after the closing brace. |
3897 | ||
342b8b6e | 3898 | @node Type Decl |
bfa74976 RS |
3899 | @subsection Nonterminal Symbols |
3900 | @cindex declaring value types, nonterminals | |
3901 | @cindex value types, nonterminals, declaring | |
3902 | @findex %type | |
3903 | ||
3904 | @noindent | |
3905 | When you use @code{%union} to specify multiple value types, you must | |
3906 | declare the value type of each nonterminal symbol for which values are | |
3907 | used. This is done with a @code{%type} declaration, like this: | |
3908 | ||
3909 | @example | |
3910 | %type <@var{type}> @var{nonterminal}@dots{} | |
3911 | @end example | |
3912 | ||
3913 | @noindent | |
704a47c4 AD |
3914 | Here @var{nonterminal} is the name of a nonterminal symbol, and |
3915 | @var{type} is the name given in the @code{%union} to the alternative | |
3916 | that you want (@pxref{Union Decl, ,The Collection of Value Types}). You | |
3917 | can give any number of nonterminal symbols in the same @code{%type} | |
3918 | declaration, if they have the same value type. Use spaces to separate | |
3919 | the symbol names. | |
bfa74976 | 3920 | |
931c7513 RS |
3921 | You can also declare the value type of a terminal symbol. To do this, |
3922 | use the same @code{<@var{type}>} construction in a declaration for the | |
3923 | terminal symbol. All kinds of token declarations allow | |
3924 | @code{<@var{type}>}. | |
3925 | ||
18d192f0 AD |
3926 | @node Initial Action Decl |
3927 | @subsection Performing Actions before Parsing | |
3928 | @findex %initial-action | |
3929 | ||
3930 | Sometimes your parser needs to perform some initializations before | |
3931 | parsing. The @code{%initial-action} directive allows for such arbitrary | |
3932 | code. | |
3933 | ||
3934 | @deffn {Directive} %initial-action @{ @var{code} @} | |
3935 | @findex %initial-action | |
287c78f6 | 3936 | Declare that the braced @var{code} must be invoked before parsing each time |
451364ed | 3937 | @code{yyparse} is called. The @var{code} may use @code{$$} and |
742e4900 | 3938 | @code{@@$} --- initial value and location of the lookahead --- and the |
451364ed | 3939 | @code{%parse-param}. |
18d192f0 AD |
3940 | @end deffn |
3941 | ||
451364ed AD |
3942 | For instance, if your locations use a file name, you may use |
3943 | ||
3944 | @example | |
48b16bbc | 3945 | %parse-param @{ char const *file_name @}; |
451364ed AD |
3946 | %initial-action |
3947 | @{ | |
4626a15d | 3948 | @@$.initialize (file_name); |
451364ed AD |
3949 | @}; |
3950 | @end example | |
3951 | ||
18d192f0 | 3952 | |
72f889cc AD |
3953 | @node Destructor Decl |
3954 | @subsection Freeing Discarded Symbols | |
3955 | @cindex freeing discarded symbols | |
3956 | @findex %destructor | |
3957 | ||
a85284cf AD |
3958 | During error recovery (@pxref{Error Recovery}), symbols already pushed |
3959 | on the stack and tokens coming from the rest of the file are discarded | |
3960 | until the parser falls on its feet. If the parser runs out of memory, | |
9d9b8b70 | 3961 | or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the |
a85284cf AD |
3962 | symbols on the stack must be discarded. Even if the parser succeeds, it |
3963 | must discard the start symbol. | |
258b75ca PE |
3964 | |
3965 | When discarded symbols convey heap based information, this memory is | |
3966 | lost. While this behavior can be tolerable for batch parsers, such as | |
4b367315 AD |
3967 | in traditional compilers, it is unacceptable for programs like shells or |
3968 | protocol implementations that may parse and execute indefinitely. | |
258b75ca | 3969 | |
a85284cf AD |
3970 | The @code{%destructor} directive defines code that is called when a |
3971 | symbol is automatically discarded. | |
72f889cc AD |
3972 | |
3973 | @deffn {Directive} %destructor @{ @var{code} @} @var{symbols} | |
3974 | @findex %destructor | |
287c78f6 PE |
3975 | Invoke the braced @var{code} whenever the parser discards one of the |
3976 | @var{symbols}. | |
4b367315 AD |
3977 | Within @var{code}, @code{$$} designates the semantic value associated |
3978 | with the discarded symbol. The additional parser parameters are also | |
3979 | available (@pxref{Parser Function, , The Parser Function | |
3980 | @code{yyparse}}). | |
72f889cc AD |
3981 | @end deffn |
3982 | ||
3983 | For instance: | |
3984 | ||
3985 | @smallexample | |
3986 | %union | |
3987 | @{ | |
3988 | char *string; | |
3989 | @} | |
3990 | %token <string> STRING | |
3991 | %type <string> string | |
3992 | %destructor @{ free ($$); @} STRING string | |
3993 | @end smallexample | |
3994 | ||
3995 | @noindent | |
258b75ca | 3996 | guarantees that when a @code{STRING} or a @code{string} is discarded, |
72f889cc AD |
3997 | its associated memory will be freed. |
3998 | ||
e757bb10 AD |
3999 | @sp 1 |
4000 | ||
4001 | @cindex discarded symbols | |
4002 | @dfn{Discarded symbols} are the following: | |
4003 | ||
4004 | @itemize | |
4005 | @item | |
4006 | stacked symbols popped during the first phase of error recovery, | |
4007 | @item | |
4008 | incoming terminals during the second phase of error recovery, | |
4009 | @item | |
742e4900 | 4010 | the current lookahead and the entire stack (except the current |
9d9b8b70 | 4011 | right-hand side symbols) when the parser returns immediately, and |
258b75ca PE |
4012 | @item |
4013 | the start symbol, when the parser succeeds. | |
e757bb10 AD |
4014 | @end itemize |
4015 | ||
9d9b8b70 PE |
4016 | The parser can @dfn{return immediately} because of an explicit call to |
4017 | @code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory | |
4018 | exhaustion. | |
4019 | ||
4020 | Right-hand size symbols of a rule that explicitly triggers a syntax | |
4021 | error via @code{YYERROR} are not discarded automatically. As a rule | |
4022 | of thumb, destructors are invoked only when user actions cannot manage | |
a85284cf | 4023 | the memory. |
e757bb10 | 4024 | |
342b8b6e | 4025 | @node Expect Decl |
bfa74976 RS |
4026 | @subsection Suppressing Conflict Warnings |
4027 | @cindex suppressing conflict warnings | |
4028 | @cindex preventing warnings about conflicts | |
4029 | @cindex warnings, preventing | |
4030 | @cindex conflicts, suppressing warnings of | |
4031 | @findex %expect | |
d6328241 | 4032 | @findex %expect-rr |
bfa74976 RS |
4033 | |
4034 | Bison normally warns if there are any conflicts in the grammar | |
7da99ede AD |
4035 | (@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars |
4036 | have harmless shift/reduce conflicts which are resolved in a predictable | |
4037 | way and would be difficult to eliminate. It is desirable to suppress | |
4038 | the warning about these conflicts unless the number of conflicts | |
4039 | changes. You can do this with the @code{%expect} declaration. | |
bfa74976 RS |
4040 | |
4041 | The declaration looks like this: | |
4042 | ||
4043 | @example | |
4044 | %expect @var{n} | |
4045 | @end example | |
4046 | ||
035aa4a0 PE |
4047 | Here @var{n} is a decimal integer. The declaration says there should |
4048 | be @var{n} shift/reduce conflicts and no reduce/reduce conflicts. | |
4049 | Bison reports an error if the number of shift/reduce conflicts differs | |
4050 | from @var{n}, or if there are any reduce/reduce conflicts. | |
bfa74976 | 4051 | |
035aa4a0 PE |
4052 | For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more |
4053 | serious, and should be eliminated entirely. Bison will always report | |
4054 | reduce/reduce conflicts for these parsers. With @acronym{GLR} | |
4055 | parsers, however, both kinds of conflicts are routine; otherwise, | |
4056 | there would be no need to use @acronym{GLR} parsing. Therefore, it is | |
4057 | also possible to specify an expected number of reduce/reduce conflicts | |
4058 | in @acronym{GLR} parsers, using the declaration: | |
d6328241 PH |
4059 | |
4060 | @example | |
4061 | %expect-rr @var{n} | |
4062 | @end example | |
4063 | ||
bfa74976 RS |
4064 | In general, using @code{%expect} involves these steps: |
4065 | ||
4066 | @itemize @bullet | |
4067 | @item | |
4068 | Compile your grammar without @code{%expect}. Use the @samp{-v} option | |
4069 | to get a verbose list of where the conflicts occur. Bison will also | |
4070 | print the number of conflicts. | |
4071 | ||
4072 | @item | |
4073 | Check each of the conflicts to make sure that Bison's default | |
4074 | resolution is what you really want. If not, rewrite the grammar and | |
4075 | go back to the beginning. | |
4076 | ||
4077 | @item | |
4078 | Add an @code{%expect} declaration, copying the number @var{n} from the | |
035aa4a0 PE |
4079 | number which Bison printed. With @acronym{GLR} parsers, add an |
4080 | @code{%expect-rr} declaration as well. | |
bfa74976 RS |
4081 | @end itemize |
4082 | ||
035aa4a0 PE |
4083 | Now Bison will warn you if you introduce an unexpected conflict, but |
4084 | will keep silent otherwise. | |
bfa74976 | 4085 | |
342b8b6e | 4086 | @node Start Decl |
bfa74976 RS |
4087 | @subsection The Start-Symbol |
4088 | @cindex declaring the start symbol | |
4089 | @cindex start symbol, declaring | |
4090 | @cindex default start symbol | |
4091 | @findex %start | |
4092 | ||
4093 | Bison assumes by default that the start symbol for the grammar is the first | |
4094 | nonterminal specified in the grammar specification section. The programmer | |
4095 | may override this restriction with the @code{%start} declaration as follows: | |
4096 | ||
4097 | @example | |
4098 | %start @var{symbol} | |
4099 | @end example | |
4100 | ||
342b8b6e | 4101 | @node Pure Decl |
bfa74976 RS |
4102 | @subsection A Pure (Reentrant) Parser |
4103 | @cindex reentrant parser | |
4104 | @cindex pure parser | |
8c9a50be | 4105 | @findex %pure-parser |
bfa74976 RS |
4106 | |
4107 | A @dfn{reentrant} program is one which does not alter in the course of | |
4108 | execution; in other words, it consists entirely of @dfn{pure} (read-only) | |
4109 | code. Reentrancy is important whenever asynchronous execution is possible; | |
9d9b8b70 PE |
4110 | for example, a nonreentrant program may not be safe to call from a signal |
4111 | handler. In systems with multiple threads of control, a nonreentrant | |
bfa74976 RS |
4112 | program must be called only within interlocks. |
4113 | ||
70811b85 | 4114 | Normally, Bison generates a parser which is not reentrant. This is |
c827f760 PE |
4115 | suitable for most uses, and it permits compatibility with Yacc. (The |
4116 | standard Yacc interfaces are inherently nonreentrant, because they use | |
70811b85 RS |
4117 | statically allocated variables for communication with @code{yylex}, |
4118 | including @code{yylval} and @code{yylloc}.) | |
bfa74976 | 4119 | |
70811b85 | 4120 | Alternatively, you can generate a pure, reentrant parser. The Bison |
8c9a50be | 4121 | declaration @code{%pure-parser} says that you want the parser to be |
70811b85 | 4122 | reentrant. It looks like this: |
bfa74976 RS |
4123 | |
4124 | @example | |
8c9a50be | 4125 | %pure-parser |
bfa74976 RS |
4126 | @end example |
4127 | ||
70811b85 RS |
4128 | The result is that the communication variables @code{yylval} and |
4129 | @code{yylloc} become local variables in @code{yyparse}, and a different | |
4130 | calling convention is used for the lexical analyzer function | |
4131 | @code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure | |
4132 | Parsers}, for the details of this. The variable @code{yynerrs} also | |
4133 | becomes local in @code{yyparse} (@pxref{Error Reporting, ,The Error | |
4134 | Reporting Function @code{yyerror}}). The convention for calling | |
4135 | @code{yyparse} itself is unchanged. | |
4136 | ||
4137 | Whether the parser is pure has nothing to do with the grammar rules. | |
4138 | You can generate either a pure parser or a nonreentrant parser from any | |
4139 | valid grammar. | |
bfa74976 | 4140 | |
342b8b6e | 4141 | @node Decl Summary |
bfa74976 RS |
4142 | @subsection Bison Declaration Summary |
4143 | @cindex Bison declaration summary | |
4144 | @cindex declaration summary | |
4145 | @cindex summary, Bison declaration | |
4146 | ||
d8988b2f | 4147 | Here is a summary of the declarations used to define a grammar: |
bfa74976 | 4148 | |
18b519c0 | 4149 | @deffn {Directive} %union |
bfa74976 RS |
4150 | Declare the collection of data types that semantic values may have |
4151 | (@pxref{Union Decl, ,The Collection of Value Types}). | |
18b519c0 | 4152 | @end deffn |
bfa74976 | 4153 | |
18b519c0 | 4154 | @deffn {Directive} %token |
bfa74976 RS |
4155 | Declare a terminal symbol (token type name) with no precedence |
4156 | or associativity specified (@pxref{Token Decl, ,Token Type Names}). | |
18b519c0 | 4157 | @end deffn |
bfa74976 | 4158 | |
18b519c0 | 4159 | @deffn {Directive} %right |
bfa74976 RS |
4160 | Declare a terminal symbol (token type name) that is right-associative |
4161 | (@pxref{Precedence Decl, ,Operator Precedence}). | |
18b519c0 | 4162 | @end deffn |
bfa74976 | 4163 | |
18b519c0 | 4164 | @deffn {Directive} %left |
bfa74976 RS |
4165 | Declare a terminal symbol (token type name) that is left-associative |
4166 | (@pxref{Precedence Decl, ,Operator Precedence}). | |
18b519c0 | 4167 | @end deffn |
bfa74976 | 4168 | |
18b519c0 | 4169 | @deffn {Directive} %nonassoc |
bfa74976 | 4170 | Declare a terminal symbol (token type name) that is nonassociative |
bfa74976 | 4171 | (@pxref{Precedence Decl, ,Operator Precedence}). |
39a06c25 PE |
4172 | Using it in a way that would be associative is a syntax error. |
4173 | @end deffn | |
4174 | ||
91d2c560 | 4175 | @ifset defaultprec |
39a06c25 | 4176 | @deffn {Directive} %default-prec |
22fccf95 | 4177 | Assign a precedence to rules lacking an explicit @code{%prec} modifier |
39a06c25 PE |
4178 | (@pxref{Contextual Precedence, ,Context-Dependent Precedence}). |
4179 | @end deffn | |
91d2c560 | 4180 | @end ifset |
bfa74976 | 4181 | |
18b519c0 | 4182 | @deffn {Directive} %type |
bfa74976 RS |
4183 | Declare the type of semantic values for a nonterminal symbol |
4184 | (@pxref{Type Decl, ,Nonterminal Symbols}). | |
18b519c0 | 4185 | @end deffn |
bfa74976 | 4186 | |
18b519c0 | 4187 | @deffn {Directive} %start |
89cab50d AD |
4188 | Specify the grammar's start symbol (@pxref{Start Decl, ,The |
4189 | Start-Symbol}). | |
18b519c0 | 4190 | @end deffn |
bfa74976 | 4191 | |
18b519c0 | 4192 | @deffn {Directive} %expect |
bfa74976 RS |
4193 | Declare the expected number of shift-reduce conflicts |
4194 | (@pxref{Expect Decl, ,Suppressing Conflict Warnings}). | |
18b519c0 AD |
4195 | @end deffn |
4196 | ||
bfa74976 | 4197 | |
d8988b2f AD |
4198 | @sp 1 |
4199 | @noindent | |
4200 | In order to change the behavior of @command{bison}, use the following | |
4201 | directives: | |
4202 | ||
18b519c0 | 4203 | @deffn {Directive} %debug |
4947ebdb PE |
4204 | In the parser file, define the macro @code{YYDEBUG} to 1 if it is not |
4205 | already defined, so that the debugging facilities are compiled. | |
18b519c0 | 4206 | @end deffn |
ec3bc396 | 4207 | @xref{Tracing, ,Tracing Your Parser}. |
d8988b2f | 4208 | |
18b519c0 | 4209 | @deffn {Directive} %defines |
4bfd5e4e PE |
4210 | Write a header file containing macro definitions for the token type |
4211 | names defined in the grammar as well as a few other declarations. | |
d8988b2f | 4212 | If the parser output file is named @file{@var{name}.c} then this file |
e0c471a9 | 4213 | is named @file{@var{name}.h}. |
d8988b2f | 4214 | |
4bfd5e4e | 4215 | Unless @code{YYSTYPE} is already defined as a macro, the output header |
5c9be03d | 4216 | declares @code{YYSTYPE}. Therefore, if you are using a @code{%union} |
f8e1c9e5 AD |
4217 | (@pxref{Multiple Types, ,More Than One Value Type}) with components that |
4218 | require other definitions, or if you have defined a @code{YYSTYPE} macro | |
4219 | (@pxref{Value Type, ,Data Types of Semantic Values}), you need to | |
4220 | arrange for these definitions to be propagated to all modules, e.g., by | |
4221 | putting them in a prerequisite header that is included both by your | |
4222 | parser and by any other module that needs @code{YYSTYPE}. | |
4bfd5e4e PE |
4223 | |
4224 | Unless your parser is pure, the output header declares @code{yylval} | |
4225 | as an external variable. @xref{Pure Decl, ,A Pure (Reentrant) | |
4226 | Parser}. | |
4227 | ||
4228 | If you have also used locations, the output header declares | |
4229 | @code{YYLTYPE} and @code{yylloc} using a protocol similar to that of | |
4230 | @code{YYSTYPE} and @code{yylval}. @xref{Locations, ,Tracking | |
4231 | Locations}. | |
4232 | ||
f8e1c9e5 AD |
4233 | This output file is normally essential if you wish to put the definition |
4234 | of @code{yylex} in a separate source file, because @code{yylex} | |
4235 | typically needs to be able to refer to the above-mentioned declarations | |
4236 | and to the token type codes. @xref{Token Values, ,Semantic Values of | |
4237 | Tokens}. | |
9bc0dd67 | 4238 | |
34f98f46 JD |
4239 | @findex %start-header |
4240 | @findex %end-header | |
4241 | If you have declared @code{%start-header} or @code{%end-header}, the output | |
4242 | header also contains their code. | |
4243 | @xref{Table of Symbols, ,%start-header}. | |
18b519c0 | 4244 | @end deffn |
d8988b2f | 4245 | |
18b519c0 | 4246 | @deffn {Directive} %destructor |
258b75ca | 4247 | Specify how the parser should reclaim the memory associated to |
fa7e68c3 | 4248 | discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}. |
18b519c0 | 4249 | @end deffn |
72f889cc | 4250 | |
18b519c0 | 4251 | @deffn {Directive} %file-prefix="@var{prefix}" |
d8988b2f AD |
4252 | Specify a prefix to use for all Bison output file names. The names are |
4253 | chosen as if the input file were named @file{@var{prefix}.y}. | |
18b519c0 | 4254 | @end deffn |
d8988b2f | 4255 | |
18b519c0 | 4256 | @deffn {Directive} %locations |
89cab50d AD |
4257 | Generate the code processing the locations (@pxref{Action Features, |
4258 | ,Special Features for Use in Actions}). This mode is enabled as soon as | |
4259 | the grammar uses the special @samp{@@@var{n}} tokens, but if your | |
4260 | grammar does not use it, using @samp{%locations} allows for more | |
6e649e65 | 4261 | accurate syntax error messages. |
18b519c0 | 4262 | @end deffn |
89cab50d | 4263 | |
18b519c0 | 4264 | @deffn {Directive} %name-prefix="@var{prefix}" |
d8988b2f AD |
4265 | Rename the external symbols used in the parser so that they start with |
4266 | @var{prefix} instead of @samp{yy}. The precise list of symbols renamed | |
aa08666d | 4267 | in C parsers |
d8988b2f | 4268 | is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs}, |
91e3ac9a PE |
4269 | @code{yylval}, @code{yychar}, @code{yydebug}, and |
4270 | (if locations are used) @code{yylloc}. For example, if you use | |
2a8d363a | 4271 | @samp{%name-prefix="c_"}, the names become @code{c_parse}, @code{c_lex}, |
aa08666d AD |
4272 | and so on. In C++ parsers, it is only the surrounding namespace which is |
4273 | named @var{prefix} instead of @samp{yy}. | |
4274 | @xref{Multiple Parsers, ,Multiple Parsers in the Same Program}. | |
18b519c0 | 4275 | @end deffn |
931c7513 | 4276 | |
91d2c560 | 4277 | @ifset defaultprec |
22fccf95 PE |
4278 | @deffn {Directive} %no-default-prec |
4279 | Do not assign a precedence to rules lacking an explicit @code{%prec} | |
4280 | modifier (@pxref{Contextual Precedence, ,Context-Dependent | |
4281 | Precedence}). | |
4282 | @end deffn | |
91d2c560 | 4283 | @end ifset |
22fccf95 | 4284 | |
18b519c0 | 4285 | @deffn {Directive} %no-parser |
6deb4447 AD |
4286 | Do not include any C code in the parser file; generate tables only. The |
4287 | parser file contains just @code{#define} directives and static variable | |
4288 | declarations. | |
4289 | ||
4290 | This option also tells Bison to write the C code for the grammar actions | |
fa4d969f | 4291 | into a file named @file{@var{file}.act}, in the form of a |
6deb4447 | 4292 | brace-surrounded body fit for a @code{switch} statement. |
18b519c0 | 4293 | @end deffn |
6deb4447 | 4294 | |
18b519c0 | 4295 | @deffn {Directive} %no-lines |
931c7513 RS |
4296 | Don't generate any @code{#line} preprocessor commands in the parser |
4297 | file. Ordinarily Bison writes these commands in the parser file so that | |
4298 | the C compiler and debuggers will associate errors and object code with | |
4299 | your source file (the grammar file). This directive causes them to | |
4300 | associate errors with the parser file, treating it an independent source | |
4301 | file in its own right. | |
18b519c0 | 4302 | @end deffn |
931c7513 | 4303 | |
fa4d969f PE |
4304 | @deffn {Directive} %output="@var{file}" |
4305 | Specify @var{file} for the parser file. | |
18b519c0 | 4306 | @end deffn |
6deb4447 | 4307 | |
18b519c0 | 4308 | @deffn {Directive} %pure-parser |
d8988b2f AD |
4309 | Request a pure (reentrant) parser program (@pxref{Pure Decl, ,A Pure |
4310 | (Reentrant) Parser}). | |
18b519c0 | 4311 | @end deffn |
6deb4447 | 4312 | |
b50d2359 | 4313 | @deffn {Directive} %require "@var{version}" |
9b8a5ce0 AD |
4314 | Require version @var{version} or higher of Bison. @xref{Require Decl, , |
4315 | Require a Version of Bison}. | |
b50d2359 AD |
4316 | @end deffn |
4317 | ||
18b519c0 | 4318 | @deffn {Directive} %token-table |
931c7513 RS |
4319 | Generate an array of token names in the parser file. The name of the |
4320 | array is @code{yytname}; @code{yytname[@var{i}]} is the name of the | |
3650b4b8 | 4321 | token whose internal Bison token code number is @var{i}. The first |
f67ad422 PE |
4322 | three elements of @code{yytname} correspond to the predefined tokens |
4323 | @code{"$end"}, | |
88bce5a2 AD |
4324 | @code{"error"}, and @code{"$undefined"}; after these come the symbols |
4325 | defined in the grammar file. | |
931c7513 | 4326 | |
9e0876fb PE |
4327 | The name in the table includes all the characters needed to represent |
4328 | the token in Bison. For single-character literals and literal | |
4329 | strings, this includes the surrounding quoting characters and any | |
4330 | escape sequences. For example, the Bison single-character literal | |
4331 | @code{'+'} corresponds to a three-character name, represented in C as | |
4332 | @code{"'+'"}; and the Bison two-character literal string @code{"\\/"} | |
4333 | corresponds to a five-character name, represented in C as | |
4334 | @code{"\"\\\\/\""}. | |
931c7513 | 4335 | |
8c9a50be | 4336 | When you specify @code{%token-table}, Bison also generates macro |
931c7513 RS |
4337 | definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and |
4338 | @code{YYNRULES}, and @code{YYNSTATES}: | |
4339 | ||
4340 | @table @code | |
4341 | @item YYNTOKENS | |
4342 | The highest token number, plus one. | |
4343 | @item YYNNTS | |
9ecbd125 | 4344 | The number of nonterminal symbols. |
931c7513 RS |
4345 | @item YYNRULES |
4346 | The number of grammar rules, | |
4347 | @item YYNSTATES | |
4348 | The number of parser states (@pxref{Parser States}). | |
4349 | @end table | |
18b519c0 | 4350 | @end deffn |
d8988b2f | 4351 | |
18b519c0 | 4352 | @deffn {Directive} %verbose |
d8988b2f | 4353 | Write an extra output file containing verbose descriptions of the |
742e4900 | 4354 | parser states and what is done for each type of lookahead token in |
72d2299c | 4355 | that state. @xref{Understanding, , Understanding Your Parser}, for more |
ec3bc396 | 4356 | information. |
18b519c0 | 4357 | @end deffn |
d8988b2f | 4358 | |
18b519c0 | 4359 | @deffn {Directive} %yacc |
d8988b2f AD |
4360 | Pretend the option @option{--yacc} was given, i.e., imitate Yacc, |
4361 | including its naming conventions. @xref{Bison Options}, for more. | |
18b519c0 | 4362 | @end deffn |
d8988b2f AD |
4363 | |
4364 | ||
342b8b6e | 4365 | @node Multiple Parsers |
bfa74976 RS |
4366 | @section Multiple Parsers in the Same Program |
4367 | ||
4368 | Most programs that use Bison parse only one language and therefore contain | |
4369 | only one Bison parser. But what if you want to parse more than one | |
4370 | language with the same program? Then you need to avoid a name conflict | |
4371 | between different definitions of @code{yyparse}, @code{yylval}, and so on. | |
4372 | ||
4373 | The easy way to do this is to use the option @samp{-p @var{prefix}} | |
704a47c4 AD |
4374 | (@pxref{Invocation, ,Invoking Bison}). This renames the interface |
4375 | functions and variables of the Bison parser to start with @var{prefix} | |
4376 | instead of @samp{yy}. You can use this to give each parser distinct | |
4377 | names that do not conflict. | |
bfa74976 RS |
4378 | |
4379 | The precise list of symbols renamed is @code{yyparse}, @code{yylex}, | |
2a8d363a AD |
4380 | @code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc}, |
4381 | @code{yychar} and @code{yydebug}. For example, if you use @samp{-p c}, | |
4382 | the names become @code{cparse}, @code{clex}, and so on. | |
bfa74976 RS |
4383 | |
4384 | @strong{All the other variables and macros associated with Bison are not | |
4385 | renamed.} These others are not global; there is no conflict if the same | |
4386 | name is used in different parsers. For example, @code{YYSTYPE} is not | |
4387 | renamed, but defining this in different ways in different parsers causes | |
4388 | no trouble (@pxref{Value Type, ,Data Types of Semantic Values}). | |
4389 | ||
4390 | The @samp{-p} option works by adding macro definitions to the beginning | |
4391 | of the parser source file, defining @code{yyparse} as | |
4392 | @code{@var{prefix}parse}, and so on. This effectively substitutes one | |
4393 | name for the other in the entire parser file. | |
4394 | ||
342b8b6e | 4395 | @node Interface |
bfa74976 RS |
4396 | @chapter Parser C-Language Interface |
4397 | @cindex C-language interface | |
4398 | @cindex interface | |
4399 | ||
4400 | The Bison parser is actually a C function named @code{yyparse}. Here we | |
4401 | describe the interface conventions of @code{yyparse} and the other | |
4402 | functions that it needs to use. | |
4403 | ||
4404 | Keep in mind that the parser uses many C identifiers starting with | |
4405 | @samp{yy} and @samp{YY} for internal purposes. If you use such an | |
75f5aaea MA |
4406 | identifier (aside from those in this manual) in an action or in epilogue |
4407 | in the grammar file, you are likely to run into trouble. | |
bfa74976 RS |
4408 | |
4409 | @menu | |
4410 | * Parser Function:: How to call @code{yyparse} and what it returns. | |
13863333 | 4411 | * Lexical:: You must supply a function @code{yylex} |
bfa74976 RS |
4412 | which reads tokens. |
4413 | * Error Reporting:: You must supply a function @code{yyerror}. | |
4414 | * Action Features:: Special features for use in actions. | |
f7ab6a50 PE |
4415 | * Internationalization:: How to let the parser speak in the user's |
4416 | native language. | |
bfa74976 RS |
4417 | @end menu |
4418 | ||
342b8b6e | 4419 | @node Parser Function |
bfa74976 RS |
4420 | @section The Parser Function @code{yyparse} |
4421 | @findex yyparse | |
4422 | ||
4423 | You call the function @code{yyparse} to cause parsing to occur. This | |
4424 | function reads tokens, executes actions, and ultimately returns when it | |
4425 | encounters end-of-input or an unrecoverable syntax error. You can also | |
14ded682 AD |
4426 | write an action which directs @code{yyparse} to return immediately |
4427 | without reading further. | |
bfa74976 | 4428 | |
2a8d363a AD |
4429 | |
4430 | @deftypefun int yyparse (void) | |
bfa74976 RS |
4431 | The value returned by @code{yyparse} is 0 if parsing was successful (return |
4432 | is due to end-of-input). | |
4433 | ||
b47dbebe PE |
4434 | The value is 1 if parsing failed because of invalid input, i.e., input |
4435 | that contains a syntax error or that causes @code{YYABORT} to be | |
4436 | invoked. | |
4437 | ||
4438 | The value is 2 if parsing failed due to memory exhaustion. | |
2a8d363a | 4439 | @end deftypefun |
bfa74976 RS |
4440 | |
4441 | In an action, you can cause immediate return from @code{yyparse} by using | |
4442 | these macros: | |
4443 | ||
2a8d363a | 4444 | @defmac YYACCEPT |
bfa74976 RS |
4445 | @findex YYACCEPT |
4446 | Return immediately with value 0 (to report success). | |
2a8d363a | 4447 | @end defmac |
bfa74976 | 4448 | |
2a8d363a | 4449 | @defmac YYABORT |
bfa74976 RS |
4450 | @findex YYABORT |
4451 | Return immediately with value 1 (to report failure). | |
2a8d363a AD |
4452 | @end defmac |
4453 | ||
4454 | If you use a reentrant parser, you can optionally pass additional | |
4455 | parameter information to it in a reentrant way. To do so, use the | |
4456 | declaration @code{%parse-param}: | |
4457 | ||
feeb0eda | 4458 | @deffn {Directive} %parse-param @{@var{argument-declaration}@} |
2a8d363a | 4459 | @findex %parse-param |
287c78f6 PE |
4460 | Declare that an argument declared by the braced-code |
4461 | @var{argument-declaration} is an additional @code{yyparse} argument. | |
94175978 | 4462 | The @var{argument-declaration} is used when declaring |
feeb0eda PE |
4463 | functions or prototypes. The last identifier in |
4464 | @var{argument-declaration} must be the argument name. | |
2a8d363a AD |
4465 | @end deffn |
4466 | ||
4467 | Here's an example. Write this in the parser: | |
4468 | ||
4469 | @example | |
feeb0eda PE |
4470 | %parse-param @{int *nastiness@} |
4471 | %parse-param @{int *randomness@} | |
2a8d363a AD |
4472 | @end example |
4473 | ||
4474 | @noindent | |
4475 | Then call the parser like this: | |
4476 | ||
4477 | @example | |
4478 | @{ | |
4479 | int nastiness, randomness; | |
4480 | @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */ | |
4481 | value = yyparse (&nastiness, &randomness); | |
4482 | @dots{} | |
4483 | @} | |
4484 | @end example | |
4485 | ||
4486 | @noindent | |
4487 | In the grammar actions, use expressions like this to refer to the data: | |
4488 | ||
4489 | @example | |
4490 | exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @} | |
4491 | @end example | |
4492 | ||
bfa74976 | 4493 | |
342b8b6e | 4494 | @node Lexical |
bfa74976 RS |
4495 | @section The Lexical Analyzer Function @code{yylex} |
4496 | @findex yylex | |
4497 | @cindex lexical analyzer | |
4498 | ||
4499 | The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from | |
4500 | the input stream and returns them to the parser. Bison does not create | |
4501 | this function automatically; you must write it so that @code{yyparse} can | |
4502 | call it. The function is sometimes referred to as a lexical scanner. | |
4503 | ||
4504 | In simple programs, @code{yylex} is often defined at the end of the Bison | |
4505 | grammar file. If @code{yylex} is defined in a separate source file, you | |
4506 | need to arrange for the token-type macro definitions to be available there. | |
4507 | To do this, use the @samp{-d} option when you run Bison, so that it will | |
4508 | write these macro definitions into a separate header file | |
4509 | @file{@var{name}.tab.h} which you can include in the other source files | |
e0c471a9 | 4510 | that need it. @xref{Invocation, ,Invoking Bison}. |
bfa74976 RS |
4511 | |
4512 | @menu | |
4513 | * Calling Convention:: How @code{yyparse} calls @code{yylex}. | |
4514 | * Token Values:: How @code{yylex} must return the semantic value | |
4515 | of the token it has read. | |
95923bd6 | 4516 | * Token Locations:: How @code{yylex} must return the text location |
bfa74976 RS |
4517 | (line number, etc.) of the token, if the |
4518 | actions want that. | |
4519 | * Pure Calling:: How the calling convention differs | |
4520 | in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). | |
4521 | @end menu | |
4522 | ||
342b8b6e | 4523 | @node Calling Convention |
bfa74976 RS |
4524 | @subsection Calling Convention for @code{yylex} |
4525 | ||
72d2299c PE |
4526 | The value that @code{yylex} returns must be the positive numeric code |
4527 | for the type of token it has just found; a zero or negative value | |
4528 | signifies end-of-input. | |
bfa74976 RS |
4529 | |
4530 | When a token is referred to in the grammar rules by a name, that name | |
4531 | in the parser file becomes a C macro whose definition is the proper | |
4532 | numeric code for that token type. So @code{yylex} can use the name | |
4533 | to indicate that type. @xref{Symbols}. | |
4534 | ||
4535 | When a token is referred to in the grammar rules by a character literal, | |
4536 | the numeric code for that character is also the code for the token type. | |
72d2299c PE |
4537 | So @code{yylex} can simply return that character code, possibly converted |
4538 | to @code{unsigned char} to avoid sign-extension. The null character | |
4539 | must not be used this way, because its code is zero and that | |
bfa74976 RS |
4540 | signifies end-of-input. |
4541 | ||
4542 | Here is an example showing these things: | |
4543 | ||
4544 | @example | |
13863333 AD |
4545 | int |
4546 | yylex (void) | |
bfa74976 RS |
4547 | @{ |
4548 | @dots{} | |
72d2299c | 4549 | if (c == EOF) /* Detect end-of-input. */ |
bfa74976 RS |
4550 | return 0; |
4551 | @dots{} | |
4552 | if (c == '+' || c == '-') | |
72d2299c | 4553 | return c; /* Assume token type for `+' is '+'. */ |
bfa74976 | 4554 | @dots{} |
72d2299c | 4555 | return INT; /* Return the type of the token. */ |
bfa74976 RS |
4556 | @dots{} |
4557 | @} | |
4558 | @end example | |
4559 | ||
4560 | @noindent | |
4561 | This interface has been designed so that the output from the @code{lex} | |
4562 | utility can be used without change as the definition of @code{yylex}. | |
4563 | ||
931c7513 RS |
4564 | If the grammar uses literal string tokens, there are two ways that |
4565 | @code{yylex} can determine the token type codes for them: | |
4566 | ||
4567 | @itemize @bullet | |
4568 | @item | |
4569 | If the grammar defines symbolic token names as aliases for the | |
4570 | literal string tokens, @code{yylex} can use these symbolic names like | |
4571 | all others. In this case, the use of the literal string tokens in | |
4572 | the grammar file has no effect on @code{yylex}. | |
4573 | ||
4574 | @item | |
9ecbd125 | 4575 | @code{yylex} can find the multicharacter token in the @code{yytname} |
931c7513 | 4576 | table. The index of the token in the table is the token type's code. |
9ecbd125 | 4577 | The name of a multicharacter token is recorded in @code{yytname} with a |
931c7513 | 4578 | double-quote, the token's characters, and another double-quote. The |
9e0876fb PE |
4579 | token's characters are escaped as necessary to be suitable as input |
4580 | to Bison. | |
931c7513 | 4581 | |
9e0876fb PE |
4582 | Here's code for looking up a multicharacter token in @code{yytname}, |
4583 | assuming that the characters of the token are stored in | |
4584 | @code{token_buffer}, and assuming that the token does not contain any | |
4585 | characters like @samp{"} that require escaping. | |
931c7513 RS |
4586 | |
4587 | @smallexample | |
4588 | for (i = 0; i < YYNTOKENS; i++) | |
4589 | @{ | |
4590 | if (yytname[i] != 0 | |
4591 | && yytname[i][0] == '"' | |
68449b3a PE |
4592 | && ! strncmp (yytname[i] + 1, token_buffer, |
4593 | strlen (token_buffer)) | |
931c7513 RS |
4594 | && yytname[i][strlen (token_buffer) + 1] == '"' |
4595 | && yytname[i][strlen (token_buffer) + 2] == 0) | |
4596 | break; | |
4597 | @} | |
4598 | @end smallexample | |
4599 | ||
4600 | The @code{yytname} table is generated only if you use the | |
8c9a50be | 4601 | @code{%token-table} declaration. @xref{Decl Summary}. |
931c7513 RS |
4602 | @end itemize |
4603 | ||
342b8b6e | 4604 | @node Token Values |
bfa74976 RS |
4605 | @subsection Semantic Values of Tokens |
4606 | ||
4607 | @vindex yylval | |
9d9b8b70 | 4608 | In an ordinary (nonreentrant) parser, the semantic value of the token must |
bfa74976 RS |
4609 | be stored into the global variable @code{yylval}. When you are using |
4610 | just one data type for semantic values, @code{yylval} has that type. | |
4611 | Thus, if the type is @code{int} (the default), you might write this in | |
4612 | @code{yylex}: | |
4613 | ||
4614 | @example | |
4615 | @group | |
4616 | @dots{} | |
72d2299c PE |
4617 | yylval = value; /* Put value onto Bison stack. */ |
4618 | return INT; /* Return the type of the token. */ | |
bfa74976 RS |
4619 | @dots{} |
4620 | @end group | |
4621 | @end example | |
4622 | ||
4623 | When you are using multiple data types, @code{yylval}'s type is a union | |
704a47c4 AD |
4624 | made from the @code{%union} declaration (@pxref{Union Decl, ,The |
4625 | Collection of Value Types}). So when you store a token's value, you | |
4626 | must use the proper member of the union. If the @code{%union} | |
4627 | declaration looks like this: | |
bfa74976 RS |
4628 | |
4629 | @example | |
4630 | @group | |
4631 | %union @{ | |
4632 | int intval; | |
4633 | double val; | |
4634 | symrec *tptr; | |
4635 | @} | |
4636 | @end group | |
4637 | @end example | |
4638 | ||
4639 | @noindent | |
4640 | then the code in @code{yylex} might look like this: | |
4641 | ||
4642 | @example | |
4643 | @group | |
4644 | @dots{} | |
72d2299c PE |
4645 | yylval.intval = value; /* Put value onto Bison stack. */ |
4646 | return INT; /* Return the type of the token. */ | |
bfa74976 RS |
4647 | @dots{} |
4648 | @end group | |
4649 | @end example | |
4650 | ||
95923bd6 AD |
4651 | @node Token Locations |
4652 | @subsection Textual Locations of Tokens | |
bfa74976 RS |
4653 | |
4654 | @vindex yylloc | |
847bf1f5 | 4655 | If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, , |
f8e1c9e5 AD |
4656 | Tracking Locations}) in actions to keep track of the textual locations |
4657 | of tokens and groupings, then you must provide this information in | |
4658 | @code{yylex}. The function @code{yyparse} expects to find the textual | |
4659 | location of a token just parsed in the global variable @code{yylloc}. | |
4660 | So @code{yylex} must store the proper data in that variable. | |
847bf1f5 AD |
4661 | |
4662 | By default, the value of @code{yylloc} is a structure and you need only | |
89cab50d AD |
4663 | initialize the members that are going to be used by the actions. The |
4664 | four members are called @code{first_line}, @code{first_column}, | |
4665 | @code{last_line} and @code{last_column}. Note that the use of this | |
4666 | feature makes the parser noticeably slower. | |
bfa74976 RS |
4667 | |
4668 | @tindex YYLTYPE | |
4669 | The data type of @code{yylloc} has the name @code{YYLTYPE}. | |
4670 | ||
342b8b6e | 4671 | @node Pure Calling |
c656404a | 4672 | @subsection Calling Conventions for Pure Parsers |
bfa74976 | 4673 | |
8c9a50be | 4674 | When you use the Bison declaration @code{%pure-parser} to request a |
e425e872 RS |
4675 | pure, reentrant parser, the global communication variables @code{yylval} |
4676 | and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant) | |
4677 | Parser}.) In such parsers the two global variables are replaced by | |
4678 | pointers passed as arguments to @code{yylex}. You must declare them as | |
4679 | shown here, and pass the information back by storing it through those | |
4680 | pointers. | |
bfa74976 RS |
4681 | |
4682 | @example | |
13863333 AD |
4683 | int |
4684 | yylex (YYSTYPE *lvalp, YYLTYPE *llocp) | |
bfa74976 RS |
4685 | @{ |
4686 | @dots{} | |
4687 | *lvalp = value; /* Put value onto Bison stack. */ | |
4688 | return INT; /* Return the type of the token. */ | |
4689 | @dots{} | |
4690 | @} | |
4691 | @end example | |
4692 | ||
4693 | If the grammar file does not use the @samp{@@} constructs to refer to | |
95923bd6 | 4694 | textual locations, then the type @code{YYLTYPE} will not be defined. In |
bfa74976 RS |
4695 | this case, omit the second argument; @code{yylex} will be called with |
4696 | only one argument. | |
4697 | ||
e425e872 | 4698 | |
2a8d363a AD |
4699 | If you wish to pass the additional parameter data to @code{yylex}, use |
4700 | @code{%lex-param} just like @code{%parse-param} (@pxref{Parser | |
4701 | Function}). | |
e425e872 | 4702 | |
feeb0eda | 4703 | @deffn {Directive} lex-param @{@var{argument-declaration}@} |
2a8d363a | 4704 | @findex %lex-param |
287c78f6 PE |
4705 | Declare that the braced-code @var{argument-declaration} is an |
4706 | additional @code{yylex} argument declaration. | |
2a8d363a | 4707 | @end deffn |
e425e872 | 4708 | |
2a8d363a | 4709 | For instance: |
e425e872 RS |
4710 | |
4711 | @example | |
feeb0eda PE |
4712 | %parse-param @{int *nastiness@} |
4713 | %lex-param @{int *nastiness@} | |
4714 | %parse-param @{int *randomness@} | |
e425e872 RS |
4715 | @end example |
4716 | ||
4717 | @noindent | |
2a8d363a | 4718 | results in the following signature: |
e425e872 RS |
4719 | |
4720 | @example | |
2a8d363a AD |
4721 | int yylex (int *nastiness); |
4722 | int yyparse (int *nastiness, int *randomness); | |
e425e872 RS |
4723 | @end example |
4724 | ||
2a8d363a | 4725 | If @code{%pure-parser} is added: |
c656404a RS |
4726 | |
4727 | @example | |
2a8d363a AD |
4728 | int yylex (YYSTYPE *lvalp, int *nastiness); |
4729 | int yyparse (int *nastiness, int *randomness); | |
c656404a RS |
4730 | @end example |
4731 | ||
2a8d363a AD |
4732 | @noindent |
4733 | and finally, if both @code{%pure-parser} and @code{%locations} are used: | |
c656404a | 4734 | |
2a8d363a AD |
4735 | @example |
4736 | int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness); | |
4737 | int yyparse (int *nastiness, int *randomness); | |
4738 | @end example | |
931c7513 | 4739 | |
342b8b6e | 4740 | @node Error Reporting |
bfa74976 RS |
4741 | @section The Error Reporting Function @code{yyerror} |
4742 | @cindex error reporting function | |
4743 | @findex yyerror | |
4744 | @cindex parse error | |
4745 | @cindex syntax error | |
4746 | ||
6e649e65 | 4747 | The Bison parser detects a @dfn{syntax error} or @dfn{parse error} |
9ecbd125 | 4748 | whenever it reads a token which cannot satisfy any syntax rule. An |
bfa74976 | 4749 | action in the grammar can also explicitly proclaim an error, using the |
ceed8467 AD |
4750 | macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use |
4751 | in Actions}). | |
bfa74976 RS |
4752 | |
4753 | The Bison parser expects to report the error by calling an error | |
4754 | reporting function named @code{yyerror}, which you must supply. It is | |
4755 | called by @code{yyparse} whenever a syntax error is found, and it | |
6e649e65 PE |
4756 | receives one argument. For a syntax error, the string is normally |
4757 | @w{@code{"syntax error"}}. | |
bfa74976 | 4758 | |
2a8d363a AD |
4759 | @findex %error-verbose |
4760 | If you invoke the directive @code{%error-verbose} in the Bison | |
4761 | declarations section (@pxref{Bison Declarations, ,The Bison Declarations | |
4762 | Section}), then Bison provides a more verbose and specific error message | |
6e649e65 | 4763 | string instead of just plain @w{@code{"syntax error"}}. |
bfa74976 | 4764 | |
1a059451 PE |
4765 | The parser can detect one other kind of error: memory exhaustion. This |
4766 | can happen when the input contains constructions that are very deeply | |
bfa74976 | 4767 | nested. It isn't likely you will encounter this, since the Bison |
1a059451 PE |
4768 | parser normally extends its stack automatically up to a very large limit. But |
4769 | if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual | |
4770 | fashion, except that the argument string is @w{@code{"memory exhausted"}}. | |
4771 | ||
4772 | In some cases diagnostics like @w{@code{"syntax error"}} are | |
4773 | translated automatically from English to some other language before | |
4774 | they are passed to @code{yyerror}. @xref{Internationalization}. | |
bfa74976 RS |
4775 | |
4776 | The following definition suffices in simple programs: | |
4777 | ||
4778 | @example | |
4779 | @group | |
13863333 | 4780 | void |
38a92d50 | 4781 | yyerror (char const *s) |
bfa74976 RS |
4782 | @{ |
4783 | @end group | |
4784 | @group | |
4785 | fprintf (stderr, "%s\n", s); | |
4786 | @} | |
4787 | @end group | |
4788 | @end example | |
4789 | ||
4790 | After @code{yyerror} returns to @code{yyparse}, the latter will attempt | |
4791 | error recovery if you have written suitable error recovery grammar rules | |
4792 | (@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will | |
4793 | immediately return 1. | |
4794 | ||
93724f13 | 4795 | Obviously, in location tracking pure parsers, @code{yyerror} should have |
fa7e68c3 PE |
4796 | an access to the current location. |
4797 | This is indeed the case for the @acronym{GLR} | |
2a8d363a AD |
4798 | parsers, but not for the Yacc parser, for historical reasons. I.e., if |
4799 | @samp{%locations %pure-parser} is passed then the prototypes for | |
4800 | @code{yyerror} are: | |
4801 | ||
4802 | @example | |
38a92d50 PE |
4803 | void yyerror (char const *msg); /* Yacc parsers. */ |
4804 | void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */ | |
2a8d363a AD |
4805 | @end example |
4806 | ||
feeb0eda | 4807 | If @samp{%parse-param @{int *nastiness@}} is used, then: |
2a8d363a AD |
4808 | |
4809 | @example | |
b317297e PE |
4810 | void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */ |
4811 | void yyerror (int *nastiness, char const *msg); /* GLR parsers. */ | |
2a8d363a AD |
4812 | @end example |
4813 | ||
fa7e68c3 | 4814 | Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling |
2a8d363a AD |
4815 | convention for absolutely pure parsers, i.e., when the calling |
4816 | convention of @code{yylex} @emph{and} the calling convention of | |
4817 | @code{%pure-parser} are pure. I.e.: | |
4818 | ||
4819 | @example | |
4820 | /* Location tracking. */ | |
4821 | %locations | |
4822 | /* Pure yylex. */ | |
4823 | %pure-parser | |
feeb0eda | 4824 | %lex-param @{int *nastiness@} |
2a8d363a | 4825 | /* Pure yyparse. */ |
feeb0eda PE |
4826 | %parse-param @{int *nastiness@} |
4827 | %parse-param @{int *randomness@} | |
2a8d363a AD |
4828 | @end example |
4829 | ||
4830 | @noindent | |
4831 | results in the following signatures for all the parser kinds: | |
4832 | ||
4833 | @example | |
4834 | int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness); | |
4835 | int yyparse (int *nastiness, int *randomness); | |
93724f13 AD |
4836 | void yyerror (YYLTYPE *locp, |
4837 | int *nastiness, int *randomness, | |
38a92d50 | 4838 | char const *msg); |
2a8d363a AD |
4839 | @end example |
4840 | ||
1c0c3e95 | 4841 | @noindent |
38a92d50 PE |
4842 | The prototypes are only indications of how the code produced by Bison |
4843 | uses @code{yyerror}. Bison-generated code always ignores the returned | |
4844 | value, so @code{yyerror} can return any type, including @code{void}. | |
4845 | Also, @code{yyerror} can be a variadic function; that is why the | |
4846 | message is always passed last. | |
4847 | ||
4848 | Traditionally @code{yyerror} returns an @code{int} that is always | |
4849 | ignored, but this is purely for historical reasons, and @code{void} is | |
4850 | preferable since it more accurately describes the return type for | |
4851 | @code{yyerror}. | |
93724f13 | 4852 | |
bfa74976 RS |
4853 | @vindex yynerrs |
4854 | The variable @code{yynerrs} contains the number of syntax errors | |
8a2800e7 | 4855 | reported so far. Normally this variable is global; but if you |
704a47c4 AD |
4856 | request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}) |
4857 | then it is a local variable which only the actions can access. | |
bfa74976 | 4858 | |
342b8b6e | 4859 | @node Action Features |
bfa74976 RS |
4860 | @section Special Features for Use in Actions |
4861 | @cindex summary, action features | |
4862 | @cindex action features summary | |
4863 | ||
4864 | Here is a table of Bison constructs, variables and macros that | |
4865 | are useful in actions. | |
4866 | ||
18b519c0 | 4867 | @deffn {Variable} $$ |
bfa74976 RS |
4868 | Acts like a variable that contains the semantic value for the |
4869 | grouping made by the current rule. @xref{Actions}. | |
18b519c0 | 4870 | @end deffn |
bfa74976 | 4871 | |
18b519c0 | 4872 | @deffn {Variable} $@var{n} |
bfa74976 RS |
4873 | Acts like a variable that contains the semantic value for the |
4874 | @var{n}th component of the current rule. @xref{Actions}. | |
18b519c0 | 4875 | @end deffn |
bfa74976 | 4876 | |
18b519c0 | 4877 | @deffn {Variable} $<@var{typealt}>$ |
bfa74976 | 4878 | Like @code{$$} but specifies alternative @var{typealt} in the union |
704a47c4 AD |
4879 | specified by the @code{%union} declaration. @xref{Action Types, ,Data |
4880 | Types of Values in Actions}. | |
18b519c0 | 4881 | @end deffn |
bfa74976 | 4882 | |
18b519c0 | 4883 | @deffn {Variable} $<@var{typealt}>@var{n} |
bfa74976 | 4884 | Like @code{$@var{n}} but specifies alternative @var{typealt} in the |
13863333 | 4885 | union specified by the @code{%union} declaration. |
e0c471a9 | 4886 | @xref{Action Types, ,Data Types of Values in Actions}. |
18b519c0 | 4887 | @end deffn |
bfa74976 | 4888 | |
18b519c0 | 4889 | @deffn {Macro} YYABORT; |
bfa74976 RS |
4890 | Return immediately from @code{yyparse}, indicating failure. |
4891 | @xref{Parser Function, ,The Parser Function @code{yyparse}}. | |
18b519c0 | 4892 | @end deffn |
bfa74976 | 4893 | |
18b519c0 | 4894 | @deffn {Macro} YYACCEPT; |
bfa74976 RS |
4895 | Return immediately from @code{yyparse}, indicating success. |
4896 | @xref{Parser Function, ,The Parser Function @code{yyparse}}. | |
18b519c0 | 4897 | @end deffn |
bfa74976 | 4898 | |
18b519c0 | 4899 | @deffn {Macro} YYBACKUP (@var{token}, @var{value}); |
bfa74976 RS |
4900 | @findex YYBACKUP |
4901 | Unshift a token. This macro is allowed only for rules that reduce | |
742e4900 | 4902 | a single value, and only when there is no lookahead token. |
c827f760 | 4903 | It is also disallowed in @acronym{GLR} parsers. |
742e4900 | 4904 | It installs a lookahead token with token type @var{token} and |
bfa74976 RS |
4905 | semantic value @var{value}; then it discards the value that was |
4906 | going to be reduced by this rule. | |
4907 | ||
4908 | If the macro is used when it is not valid, such as when there is | |
742e4900 | 4909 | a lookahead token already, then it reports a syntax error with |
bfa74976 RS |
4910 | a message @samp{cannot back up} and performs ordinary error |
4911 | recovery. | |
4912 | ||
4913 | In either case, the rest of the action is not executed. | |
18b519c0 | 4914 | @end deffn |
bfa74976 | 4915 | |
18b519c0 | 4916 | @deffn {Macro} YYEMPTY |
bfa74976 | 4917 | @vindex YYEMPTY |
742e4900 | 4918 | Value stored in @code{yychar} when there is no lookahead token. |
18b519c0 | 4919 | @end deffn |
bfa74976 | 4920 | |
32c29292 JD |
4921 | @deffn {Macro} YYEOF |
4922 | @vindex YYEOF | |
742e4900 | 4923 | Value stored in @code{yychar} when the lookahead is the end of the input |
32c29292 JD |
4924 | stream. |
4925 | @end deffn | |
4926 | ||
18b519c0 | 4927 | @deffn {Macro} YYERROR; |
bfa74976 RS |
4928 | @findex YYERROR |
4929 | Cause an immediate syntax error. This statement initiates error | |
4930 | recovery just as if the parser itself had detected an error; however, it | |
4931 | does not call @code{yyerror}, and does not print any message. If you | |
4932 | want to print an error message, call @code{yyerror} explicitly before | |
4933 | the @samp{YYERROR;} statement. @xref{Error Recovery}. | |
18b519c0 | 4934 | @end deffn |
bfa74976 | 4935 | |
18b519c0 | 4936 | @deffn {Macro} YYRECOVERING |
02103984 PE |
4937 | @findex YYRECOVERING |
4938 | The expression @code{YYRECOVERING ()} yields 1 when the parser | |
4939 | is recovering from a syntax error, and 0 otherwise. | |
bfa74976 | 4940 | @xref{Error Recovery}. |
18b519c0 | 4941 | @end deffn |
bfa74976 | 4942 | |
18b519c0 | 4943 | @deffn {Variable} yychar |
742e4900 JD |
4944 | Variable containing either the lookahead token, or @code{YYEOF} when the |
4945 | lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead | |
32c29292 JD |
4946 | has been performed so the next token is not yet known. |
4947 | Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic | |
4948 | Actions}). | |
742e4900 | 4949 | @xref{Lookahead, ,Lookahead Tokens}. |
18b519c0 | 4950 | @end deffn |
bfa74976 | 4951 | |
18b519c0 | 4952 | @deffn {Macro} yyclearin; |
742e4900 | 4953 | Discard the current lookahead token. This is useful primarily in |
32c29292 JD |
4954 | error rules. |
4955 | Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR | |
4956 | Semantic Actions}). | |
4957 | @xref{Error Recovery}. | |
18b519c0 | 4958 | @end deffn |
bfa74976 | 4959 | |
18b519c0 | 4960 | @deffn {Macro} yyerrok; |
bfa74976 | 4961 | Resume generating error messages immediately for subsequent syntax |
13863333 | 4962 | errors. This is useful primarily in error rules. |
bfa74976 | 4963 | @xref{Error Recovery}. |
18b519c0 | 4964 | @end deffn |
bfa74976 | 4965 | |
32c29292 | 4966 | @deffn {Variable} yylloc |
742e4900 | 4967 | Variable containing the lookahead token location when @code{yychar} is not set |
32c29292 JD |
4968 | to @code{YYEMPTY} or @code{YYEOF}. |
4969 | Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic | |
4970 | Actions}). | |
4971 | @xref{Actions and Locations, ,Actions and Locations}. | |
4972 | @end deffn | |
4973 | ||
4974 | @deffn {Variable} yylval | |
742e4900 | 4975 | Variable containing the lookahead token semantic value when @code{yychar} is |
32c29292 JD |
4976 | not set to @code{YYEMPTY} or @code{YYEOF}. |
4977 | Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic | |
4978 | Actions}). | |
4979 | @xref{Actions, ,Actions}. | |
4980 | @end deffn | |
4981 | ||
18b519c0 | 4982 | @deffn {Value} @@$ |
847bf1f5 | 4983 | @findex @@$ |
95923bd6 | 4984 | Acts like a structure variable containing information on the textual location |
847bf1f5 AD |
4985 | of the grouping made by the current rule. @xref{Locations, , |
4986 | Tracking Locations}. | |
bfa74976 | 4987 | |
847bf1f5 AD |
4988 | @c Check if those paragraphs are still useful or not. |
4989 | ||
4990 | @c @example | |
4991 | @c struct @{ | |
4992 | @c int first_line, last_line; | |
4993 | @c int first_column, last_column; | |
4994 | @c @}; | |
4995 | @c @end example | |
4996 | ||
4997 | @c Thus, to get the starting line number of the third component, you would | |
4998 | @c use @samp{@@3.first_line}. | |
bfa74976 | 4999 | |
847bf1f5 AD |
5000 | @c In order for the members of this structure to contain valid information, |
5001 | @c you must make @code{yylex} supply this information about each token. | |
5002 | @c If you need only certain members, then @code{yylex} need only fill in | |
5003 | @c those members. | |
bfa74976 | 5004 | |
847bf1f5 | 5005 | @c The use of this feature makes the parser noticeably slower. |
18b519c0 | 5006 | @end deffn |
847bf1f5 | 5007 | |
18b519c0 | 5008 | @deffn {Value} @@@var{n} |
847bf1f5 | 5009 | @findex @@@var{n} |
95923bd6 | 5010 | Acts like a structure variable containing information on the textual location |
847bf1f5 AD |
5011 | of the @var{n}th component of the current rule. @xref{Locations, , |
5012 | Tracking Locations}. | |
18b519c0 | 5013 | @end deffn |
bfa74976 | 5014 | |
f7ab6a50 PE |
5015 | @node Internationalization |
5016 | @section Parser Internationalization | |
5017 | @cindex internationalization | |
5018 | @cindex i18n | |
5019 | @cindex NLS | |
5020 | @cindex gettext | |
5021 | @cindex bison-po | |
5022 | ||
5023 | A Bison-generated parser can print diagnostics, including error and | |
5024 | tracing messages. By default, they appear in English. However, Bison | |
f8e1c9e5 AD |
5025 | also supports outputting diagnostics in the user's native language. To |
5026 | make this work, the user should set the usual environment variables. | |
5027 | @xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}. | |
5028 | For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might | |
5029 | set the user's locale to French Canadian using the @acronym{UTF}-8 | |
f7ab6a50 PE |
5030 | encoding. The exact set of available locales depends on the user's |
5031 | installation. | |
5032 | ||
5033 | The maintainer of a package that uses a Bison-generated parser enables | |
5034 | the internationalization of the parser's output through the following | |
5035 | steps. Here we assume a package that uses @acronym{GNU} Autoconf and | |
5036 | @acronym{GNU} Automake. | |
5037 | ||
5038 | @enumerate | |
5039 | @item | |
30757c8c | 5040 | @cindex bison-i18n.m4 |
f7ab6a50 PE |
5041 | Into the directory containing the @acronym{GNU} Autoconf macros used |
5042 | by the package---often called @file{m4}---copy the | |
5043 | @file{bison-i18n.m4} file installed by Bison under | |
5044 | @samp{share/aclocal/bison-i18n.m4} in Bison's installation directory. | |
5045 | For example: | |
5046 | ||
5047 | @example | |
5048 | cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4 | |
5049 | @end example | |
5050 | ||
5051 | @item | |
30757c8c PE |
5052 | @findex BISON_I18N |
5053 | @vindex BISON_LOCALEDIR | |
5054 | @vindex YYENABLE_NLS | |
f7ab6a50 PE |
5055 | In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT} |
5056 | invocation, add an invocation of @code{BISON_I18N}. This macro is | |
5057 | defined in the file @file{bison-i18n.m4} that you copied earlier. It | |
5058 | causes @samp{configure} to find the value of the | |
30757c8c PE |
5059 | @code{BISON_LOCALEDIR} variable, and it defines the source-language |
5060 | symbol @code{YYENABLE_NLS} to enable translations in the | |
5061 | Bison-generated parser. | |
f7ab6a50 PE |
5062 | |
5063 | @item | |
5064 | In the @code{main} function of your program, designate the directory | |
5065 | containing Bison's runtime message catalog, through a call to | |
5066 | @samp{bindtextdomain} with domain name @samp{bison-runtime}. | |
5067 | For example: | |
5068 | ||
5069 | @example | |
5070 | bindtextdomain ("bison-runtime", BISON_LOCALEDIR); | |
5071 | @end example | |
5072 | ||
5073 | Typically this appears after any other call @code{bindtextdomain | |
5074 | (PACKAGE, LOCALEDIR)} that your package already has. Here we rely on | |
5075 | @samp{BISON_LOCALEDIR} to be defined as a string through the | |
5076 | @file{Makefile}. | |
5077 | ||
5078 | @item | |
5079 | In the @file{Makefile.am} that controls the compilation of the @code{main} | |
5080 | function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro, | |
5081 | either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example: | |
5082 | ||
5083 | @example | |
5084 | DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"' | |
5085 | @end example | |
5086 | ||
5087 | or: | |
5088 | ||
5089 | @example | |
5090 | AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"' | |
5091 | @end example | |
5092 | ||
5093 | @item | |
5094 | Finally, invoke the command @command{autoreconf} to generate the build | |
5095 | infrastructure. | |
5096 | @end enumerate | |
5097 | ||
bfa74976 | 5098 | |
342b8b6e | 5099 | @node Algorithm |
13863333 AD |
5100 | @chapter The Bison Parser Algorithm |
5101 | @cindex Bison parser algorithm | |
bfa74976 RS |
5102 | @cindex algorithm of parser |
5103 | @cindex shifting | |
5104 | @cindex reduction | |
5105 | @cindex parser stack | |
5106 | @cindex stack, parser | |
5107 | ||
5108 | As Bison reads tokens, it pushes them onto a stack along with their | |
5109 | semantic values. The stack is called the @dfn{parser stack}. Pushing a | |
5110 | token is traditionally called @dfn{shifting}. | |
5111 | ||
5112 | For example, suppose the infix calculator has read @samp{1 + 5 *}, with a | |
5113 | @samp{3} to come. The stack will have four elements, one for each token | |
5114 | that was shifted. | |
5115 | ||
5116 | But the stack does not always have an element for each token read. When | |
5117 | the last @var{n} tokens and groupings shifted match the components of a | |
5118 | grammar rule, they can be combined according to that rule. This is called | |
5119 | @dfn{reduction}. Those tokens and groupings are replaced on the stack by a | |
5120 | single grouping whose symbol is the result (left hand side) of that rule. | |
5121 | Running the rule's action is part of the process of reduction, because this | |
5122 | is what computes the semantic value of the resulting grouping. | |
5123 | ||
5124 | For example, if the infix calculator's parser stack contains this: | |
5125 | ||
5126 | @example | |
5127 | 1 + 5 * 3 | |
5128 | @end example | |
5129 | ||
5130 | @noindent | |
5131 | and the next input token is a newline character, then the last three | |
5132 | elements can be reduced to 15 via the rule: | |
5133 | ||
5134 | @example | |
5135 | expr: expr '*' expr; | |
5136 | @end example | |
5137 | ||
5138 | @noindent | |
5139 | Then the stack contains just these three elements: | |
5140 | ||
5141 | @example | |
5142 | 1 + 15 | |
5143 | @end example | |
5144 | ||
5145 | @noindent | |
5146 | At this point, another reduction can be made, resulting in the single value | |
5147 | 16. Then the newline token can be shifted. | |
5148 | ||
5149 | The parser tries, by shifts and reductions, to reduce the entire input down | |
5150 | to a single grouping whose symbol is the grammar's start-symbol | |
5151 | (@pxref{Language and Grammar, ,Languages and Context-Free Grammars}). | |
5152 | ||
5153 | This kind of parser is known in the literature as a bottom-up parser. | |
5154 | ||
5155 | @menu | |
742e4900 | 5156 | * Lookahead:: Parser looks one token ahead when deciding what to do. |
bfa74976 RS |
5157 | * Shift/Reduce:: Conflicts: when either shifting or reduction is valid. |
5158 | * Precedence:: Operator precedence works by resolving conflicts. | |
5159 | * Contextual Precedence:: When an operator's precedence depends on context. | |
5160 | * Parser States:: The parser is a finite-state-machine with stack. | |
5161 | * Reduce/Reduce:: When two rules are applicable in the same situation. | |
5162 | * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified. | |
676385e2 | 5163 | * Generalized LR Parsing:: Parsing arbitrary context-free grammars. |
1a059451 | 5164 | * Memory Management:: What happens when memory is exhausted. How to avoid it. |
bfa74976 RS |
5165 | @end menu |
5166 | ||
742e4900 JD |
5167 | @node Lookahead |
5168 | @section Lookahead Tokens | |
5169 | @cindex lookahead token | |
bfa74976 RS |
5170 | |
5171 | The Bison parser does @emph{not} always reduce immediately as soon as the | |
5172 | last @var{n} tokens and groupings match a rule. This is because such a | |
5173 | simple strategy is inadequate to handle most languages. Instead, when a | |
5174 | reduction is possible, the parser sometimes ``looks ahead'' at the next | |
5175 | token in order to decide what to do. | |
5176 | ||
5177 | When a token is read, it is not immediately shifted; first it becomes the | |
742e4900 | 5178 | @dfn{lookahead token}, which is not on the stack. Now the parser can |
bfa74976 | 5179 | perform one or more reductions of tokens and groupings on the stack, while |
742e4900 JD |
5180 | the lookahead token remains off to the side. When no more reductions |
5181 | should take place, the lookahead token is shifted onto the stack. This | |
bfa74976 | 5182 | does not mean that all possible reductions have been done; depending on the |
742e4900 | 5183 | token type of the lookahead token, some rules may choose to delay their |
bfa74976 RS |
5184 | application. |
5185 | ||
742e4900 | 5186 | Here is a simple case where lookahead is needed. These three rules define |
bfa74976 RS |
5187 | expressions which contain binary addition operators and postfix unary |
5188 | factorial operators (@samp{!}), and allow parentheses for grouping. | |
5189 | ||
5190 | @example | |
5191 | @group | |
5192 | expr: term '+' expr | |
5193 | | term | |
5194 | ; | |
5195 | @end group | |
5196 | ||
5197 | @group | |
5198 | term: '(' expr ')' | |
5199 | | term '!' | |
5200 | | NUMBER | |
5201 | ; | |
5202 | @end group | |
5203 | @end example | |
5204 | ||
5205 | Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what | |
5206 | should be done? If the following token is @samp{)}, then the first three | |
5207 | tokens must be reduced to form an @code{expr}. This is the only valid | |
5208 | course, because shifting the @samp{)} would produce a sequence of symbols | |
5209 | @w{@code{term ')'}}, and no rule allows this. | |
5210 | ||
5211 | If the following token is @samp{!}, then it must be shifted immediately so | |
5212 | that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the | |
5213 | parser were to reduce before shifting, @w{@samp{1 + 2}} would become an | |
5214 | @code{expr}. It would then be impossible to shift the @samp{!} because | |
5215 | doing so would produce on the stack the sequence of symbols @code{expr | |
5216 | '!'}. No rule allows that sequence. | |
5217 | ||
5218 | @vindex yychar | |
32c29292 JD |
5219 | @vindex yylval |
5220 | @vindex yylloc | |
742e4900 | 5221 | The lookahead token is stored in the variable @code{yychar}. |
32c29292 JD |
5222 | Its semantic value and location, if any, are stored in the variables |
5223 | @code{yylval} and @code{yylloc}. | |
bfa74976 RS |
5224 | @xref{Action Features, ,Special Features for Use in Actions}. |
5225 | ||
342b8b6e | 5226 | @node Shift/Reduce |
bfa74976 RS |
5227 | @section Shift/Reduce Conflicts |
5228 | @cindex conflicts | |
5229 | @cindex shift/reduce conflicts | |
5230 | @cindex dangling @code{else} | |
5231 | @cindex @code{else}, dangling | |
5232 | ||
5233 | Suppose we are parsing a language which has if-then and if-then-else | |
5234 | statements, with a pair of rules like this: | |
5235 | ||
5236 | @example | |
5237 | @group | |
5238 | if_stmt: | |
5239 | IF expr THEN stmt | |
5240 | | IF expr THEN stmt ELSE stmt | |
5241 | ; | |
5242 | @end group | |
5243 | @end example | |
5244 | ||
5245 | @noindent | |
5246 | Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are | |
5247 | terminal symbols for specific keyword tokens. | |
5248 | ||
742e4900 | 5249 | When the @code{ELSE} token is read and becomes the lookahead token, the |
bfa74976 RS |
5250 | contents of the stack (assuming the input is valid) are just right for |
5251 | reduction by the first rule. But it is also legitimate to shift the | |
5252 | @code{ELSE}, because that would lead to eventual reduction by the second | |
5253 | rule. | |
5254 | ||
5255 | This situation, where either a shift or a reduction would be valid, is | |
5256 | called a @dfn{shift/reduce conflict}. Bison is designed to resolve | |
5257 | these conflicts by choosing to shift, unless otherwise directed by | |
5258 | operator precedence declarations. To see the reason for this, let's | |
5259 | contrast it with the other alternative. | |
5260 | ||
5261 | Since the parser prefers to shift the @code{ELSE}, the result is to attach | |
5262 | the else-clause to the innermost if-statement, making these two inputs | |
5263 | equivalent: | |
5264 | ||
5265 | @example | |
5266 | if x then if y then win (); else lose; | |
5267 | ||
5268 | if x then do; if y then win (); else lose; end; | |
5269 | @end example | |
5270 | ||
5271 | But if the parser chose to reduce when possible rather than shift, the | |
5272 | result would be to attach the else-clause to the outermost if-statement, | |
5273 | making these two inputs equivalent: | |
5274 | ||
5275 | @example | |
5276 | if x then if y then win (); else lose; | |
5277 | ||
5278 | if x then do; if y then win (); end; else lose; | |
5279 | @end example | |
5280 | ||
5281 | The conflict exists because the grammar as written is ambiguous: either | |
5282 | parsing of the simple nested if-statement is legitimate. The established | |
5283 | convention is that these ambiguities are resolved by attaching the | |
5284 | else-clause to the innermost if-statement; this is what Bison accomplishes | |
5285 | by choosing to shift rather than reduce. (It would ideally be cleaner to | |
5286 | write an unambiguous grammar, but that is very hard to do in this case.) | |
5287 | This particular ambiguity was first encountered in the specifications of | |
5288 | Algol 60 and is called the ``dangling @code{else}'' ambiguity. | |
5289 | ||
5290 | To avoid warnings from Bison about predictable, legitimate shift/reduce | |
5291 | conflicts, use the @code{%expect @var{n}} declaration. There will be no | |
5292 | warning as long as the number of shift/reduce conflicts is exactly @var{n}. | |
5293 | @xref{Expect Decl, ,Suppressing Conflict Warnings}. | |
5294 | ||
5295 | The definition of @code{if_stmt} above is solely to blame for the | |
5296 | conflict, but the conflict does not actually appear without additional | |
5297 | rules. Here is a complete Bison input file that actually manifests the | |
5298 | conflict: | |
5299 | ||
5300 | @example | |
5301 | @group | |
5302 | %token IF THEN ELSE variable | |
5303 | %% | |
5304 | @end group | |
5305 | @group | |
5306 | stmt: expr | |
5307 | | if_stmt | |
5308 | ; | |
5309 | @end group | |
5310 | ||
5311 | @group | |
5312 | if_stmt: | |
5313 | IF expr THEN stmt | |
5314 | | IF expr THEN stmt ELSE stmt | |
5315 | ; | |
5316 | @end group | |
5317 | ||
5318 | expr: variable | |
5319 | ; | |
5320 | @end example | |
5321 | ||
342b8b6e | 5322 | @node Precedence |
bfa74976 RS |
5323 | @section Operator Precedence |
5324 | @cindex operator precedence | |
5325 | @cindex precedence of operators | |
5326 | ||
5327 | Another situation where shift/reduce conflicts appear is in arithmetic | |
5328 | expressions. Here shifting is not always the preferred resolution; the | |
5329 | Bison declarations for operator precedence allow you to specify when to | |
5330 | shift and when to reduce. | |
5331 | ||
5332 | @menu | |
5333 | * Why Precedence:: An example showing why precedence is needed. | |
5334 | * Using Precedence:: How to specify precedence in Bison grammars. | |
5335 | * Precedence Examples:: How these features are used in the previous example. | |
5336 | * How Precedence:: How they work. | |
5337 | @end menu | |
5338 | ||
342b8b6e | 5339 | @node Why Precedence |
bfa74976 RS |
5340 | @subsection When Precedence is Needed |
5341 | ||
5342 | Consider the following ambiguous grammar fragment (ambiguous because the | |
5343 | input @w{@samp{1 - 2 * 3}} can be parsed in two different ways): | |
5344 | ||
5345 | @example | |
5346 | @group | |
5347 | expr: expr '-' expr | |
5348 | | expr '*' expr | |
5349 | | expr '<' expr | |
5350 | | '(' expr ')' | |
5351 | @dots{} | |
5352 | ; | |
5353 | @end group | |
5354 | @end example | |
5355 | ||
5356 | @noindent | |
5357 | Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2}; | |
14ded682 AD |
5358 | should it reduce them via the rule for the subtraction operator? It |
5359 | depends on the next token. Of course, if the next token is @samp{)}, we | |
5360 | must reduce; shifting is invalid because no single rule can reduce the | |
5361 | token sequence @w{@samp{- 2 )}} or anything starting with that. But if | |
5362 | the next token is @samp{*} or @samp{<}, we have a choice: either | |
5363 | shifting or reduction would allow the parse to complete, but with | |
5364 | different results. | |
5365 | ||
5366 | To decide which one Bison should do, we must consider the results. If | |
5367 | the next operator token @var{op} is shifted, then it must be reduced | |
5368 | first in order to permit another opportunity to reduce the difference. | |
5369 | The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other | |
5370 | hand, if the subtraction is reduced before shifting @var{op}, the result | |
5371 | is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or | |
5372 | reduce should depend on the relative precedence of the operators | |
5373 | @samp{-} and @var{op}: @samp{*} should be shifted first, but not | |
5374 | @samp{<}. | |
bfa74976 RS |
5375 | |
5376 | @cindex associativity | |
5377 | What about input such as @w{@samp{1 - 2 - 5}}; should this be | |
14ded682 AD |
5378 | @w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most |
5379 | operators we prefer the former, which is called @dfn{left association}. | |
5380 | The latter alternative, @dfn{right association}, is desirable for | |
5381 | assignment operators. The choice of left or right association is a | |
5382 | matter of whether the parser chooses to shift or reduce when the stack | |
742e4900 | 5383 | contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting |
14ded682 | 5384 | makes right-associativity. |
bfa74976 | 5385 | |
342b8b6e | 5386 | @node Using Precedence |
bfa74976 RS |
5387 | @subsection Specifying Operator Precedence |
5388 | @findex %left | |
5389 | @findex %right | |
5390 | @findex %nonassoc | |
5391 | ||
5392 | Bison allows you to specify these choices with the operator precedence | |
5393 | declarations @code{%left} and @code{%right}. Each such declaration | |
5394 | contains a list of tokens, which are operators whose precedence and | |
5395 | associativity is being declared. The @code{%left} declaration makes all | |
5396 | those operators left-associative and the @code{%right} declaration makes | |
5397 | them right-associative. A third alternative is @code{%nonassoc}, which | |
5398 | declares that it is a syntax error to find the same operator twice ``in a | |
5399 | row''. | |
5400 | ||
5401 | The relative precedence of different operators is controlled by the | |
5402 | order in which they are declared. The first @code{%left} or | |
5403 | @code{%right} declaration in the file declares the operators whose | |
5404 | precedence is lowest, the next such declaration declares the operators | |
5405 | whose precedence is a little higher, and so on. | |
5406 | ||
342b8b6e | 5407 | @node Precedence Examples |
bfa74976 RS |
5408 | @subsection Precedence Examples |
5409 | ||
5410 | In our example, we would want the following declarations: | |
5411 | ||
5412 | @example | |
5413 | %left '<' | |
5414 | %left '-' | |
5415 | %left '*' | |
5416 | @end example | |
5417 | ||
5418 | In a more complete example, which supports other operators as well, we | |
5419 | would declare them in groups of equal precedence. For example, @code{'+'} is | |
5420 | declared with @code{'-'}: | |
5421 | ||
5422 | @example | |
5423 | %left '<' '>' '=' NE LE GE | |
5424 | %left '+' '-' | |
5425 | %left '*' '/' | |
5426 | @end example | |
5427 | ||
5428 | @noindent | |
5429 | (Here @code{NE} and so on stand for the operators for ``not equal'' | |
5430 | and so on. We assume that these tokens are more than one character long | |
5431 | and therefore are represented by names, not character literals.) | |
5432 | ||
342b8b6e | 5433 | @node How Precedence |
bfa74976 RS |
5434 | @subsection How Precedence Works |
5435 | ||
5436 | The first effect of the precedence declarations is to assign precedence | |
5437 | levels to the terminal symbols declared. The second effect is to assign | |
704a47c4 AD |
5438 | precedence levels to certain rules: each rule gets its precedence from |
5439 | the last terminal symbol mentioned in the components. (You can also | |
5440 | specify explicitly the precedence of a rule. @xref{Contextual | |
5441 | Precedence, ,Context-Dependent Precedence}.) | |
5442 | ||
5443 | Finally, the resolution of conflicts works by comparing the precedence | |
742e4900 | 5444 | of the rule being considered with that of the lookahead token. If the |
704a47c4 AD |
5445 | token's precedence is higher, the choice is to shift. If the rule's |
5446 | precedence is higher, the choice is to reduce. If they have equal | |
5447 | precedence, the choice is made based on the associativity of that | |
5448 | precedence level. The verbose output file made by @samp{-v} | |
5449 | (@pxref{Invocation, ,Invoking Bison}) says how each conflict was | |
5450 | resolved. | |
bfa74976 RS |
5451 | |
5452 | Not all rules and not all tokens have precedence. If either the rule or | |
742e4900 | 5453 | the lookahead token has no precedence, then the default is to shift. |
bfa74976 | 5454 | |
342b8b6e | 5455 | @node Contextual Precedence |
bfa74976 RS |
5456 | @section Context-Dependent Precedence |
5457 | @cindex context-dependent precedence | |
5458 | @cindex unary operator precedence | |
5459 | @cindex precedence, context-dependent | |
5460 | @cindex precedence, unary operator | |
5461 | @findex %prec | |
5462 | ||
5463 | Often the precedence of an operator depends on the context. This sounds | |
5464 | outlandish at first, but it is really very common. For example, a minus | |
5465 | sign typically has a very high precedence as a unary operator, and a | |
5466 | somewhat lower precedence (lower than multiplication) as a binary operator. | |
5467 | ||
5468 | The Bison precedence declarations, @code{%left}, @code{%right} and | |
5469 | @code{%nonassoc}, can only be used once for a given token; so a token has | |
5470 | only one precedence declared in this way. For context-dependent | |
5471 | precedence, you need to use an additional mechanism: the @code{%prec} | |
e0c471a9 | 5472 | modifier for rules. |
bfa74976 RS |
5473 | |
5474 | The @code{%prec} modifier declares the precedence of a particular rule by | |
5475 | specifying a terminal symbol whose precedence should be used for that rule. | |
5476 | It's not necessary for that symbol to appear otherwise in the rule. The | |
5477 | modifier's syntax is: | |
5478 | ||
5479 | @example | |
5480 | %prec @var{terminal-symbol} | |
5481 | @end example | |
5482 | ||
5483 | @noindent | |
5484 | and it is written after the components of the rule. Its effect is to | |
5485 | assign the rule the precedence of @var{terminal-symbol}, overriding | |
5486 | the precedence that would be deduced for it in the ordinary way. The | |
5487 | altered rule precedence then affects how conflicts involving that rule | |
5488 | are resolved (@pxref{Precedence, ,Operator Precedence}). | |
5489 | ||
5490 | Here is how @code{%prec} solves the problem of unary minus. First, declare | |
5491 | a precedence for a fictitious terminal symbol named @code{UMINUS}. There | |
5492 | are no tokens of this type, but the symbol serves to stand for its | |
5493 | precedence: | |
5494 | ||
5495 | @example | |
5496 | @dots{} | |
5497 | %left '+' '-' | |
5498 | %left '*' | |
5499 | %left UMINUS | |
5500 | @end example | |
5501 | ||
5502 | Now the precedence of @code{UMINUS} can be used in specific rules: | |
5503 | ||
5504 | @example | |
5505 | @group | |
5506 | exp: @dots{} | |
5507 | | exp '-' exp | |
5508 | @dots{} | |
5509 | | '-' exp %prec UMINUS | |
5510 | @end group | |
5511 | @end example | |
5512 | ||
91d2c560 | 5513 | @ifset defaultprec |
39a06c25 PE |
5514 | If you forget to append @code{%prec UMINUS} to the rule for unary |
5515 | minus, Bison silently assumes that minus has its usual precedence. | |
5516 | This kind of problem can be tricky to debug, since one typically | |
5517 | discovers the mistake only by testing the code. | |
5518 | ||
22fccf95 | 5519 | The @code{%no-default-prec;} declaration makes it easier to discover |
39a06c25 PE |
5520 | this kind of problem systematically. It causes rules that lack a |
5521 | @code{%prec} modifier to have no precedence, even if the last terminal | |
5522 | symbol mentioned in their components has a declared precedence. | |
5523 | ||
22fccf95 | 5524 | If @code{%no-default-prec;} is in effect, you must specify @code{%prec} |
39a06c25 PE |
5525 | for all rules that participate in precedence conflict resolution. |
5526 | Then you will see any shift/reduce conflict until you tell Bison how | |
5527 | to resolve it, either by changing your grammar or by adding an | |
5528 | explicit precedence. This will probably add declarations to the | |
5529 | grammar, but it helps to protect against incorrect rule precedences. | |
5530 | ||
22fccf95 PE |
5531 | The effect of @code{%no-default-prec;} can be reversed by giving |
5532 | @code{%default-prec;}, which is the default. | |
91d2c560 | 5533 | @end ifset |
39a06c25 | 5534 | |
342b8b6e | 5535 | @node Parser States |
bfa74976 RS |
5536 | @section Parser States |
5537 | @cindex finite-state machine | |
5538 | @cindex parser state | |
5539 | @cindex state (of parser) | |
5540 | ||
5541 | The function @code{yyparse} is implemented using a finite-state machine. | |
5542 | The values pushed on the parser stack are not simply token type codes; they | |
5543 | represent the entire sequence of terminal and nonterminal symbols at or | |
5544 | near the top of the stack. The current state collects all the information | |
5545 | about previous input which is relevant to deciding what to do next. | |
5546 | ||
742e4900 JD |
5547 | Each time a lookahead token is read, the current parser state together |
5548 | with the type of lookahead token are looked up in a table. This table | |
5549 | entry can say, ``Shift the lookahead token.'' In this case, it also | |
bfa74976 RS |
5550 | specifies the new parser state, which is pushed onto the top of the |
5551 | parser stack. Or it can say, ``Reduce using rule number @var{n}.'' | |
5552 | This means that a certain number of tokens or groupings are taken off | |
5553 | the top of the stack, and replaced by one grouping. In other words, | |
5554 | that number of states are popped from the stack, and one new state is | |
5555 | pushed. | |
5556 | ||
742e4900 | 5557 | There is one other alternative: the table can say that the lookahead token |
bfa74976 RS |
5558 | is erroneous in the current state. This causes error processing to begin |
5559 | (@pxref{Error Recovery}). | |
5560 | ||
342b8b6e | 5561 | @node Reduce/Reduce |
bfa74976 RS |
5562 | @section Reduce/Reduce Conflicts |
5563 | @cindex reduce/reduce conflict | |
5564 | @cindex conflicts, reduce/reduce | |
5565 | ||
5566 | A reduce/reduce conflict occurs if there are two or more rules that apply | |
5567 | to the same sequence of input. This usually indicates a serious error | |
5568 | in the grammar. | |
5569 | ||
5570 | For example, here is an erroneous attempt to define a sequence | |
5571 | of zero or more @code{word} groupings. | |
5572 | ||
5573 | @example | |
5574 | sequence: /* empty */ | |
5575 | @{ printf ("empty sequence\n"); @} | |
5576 | | maybeword | |
5577 | | sequence word | |
5578 | @{ printf ("added word %s\n", $2); @} | |
5579 | ; | |
5580 | ||
5581 | maybeword: /* empty */ | |
5582 | @{ printf ("empty maybeword\n"); @} | |
5583 | | word | |
5584 | @{ printf ("single word %s\n", $1); @} | |
5585 | ; | |
5586 | @end example | |
5587 | ||
5588 | @noindent | |
5589 | The error is an ambiguity: there is more than one way to parse a single | |
5590 | @code{word} into a @code{sequence}. It could be reduced to a | |
5591 | @code{maybeword} and then into a @code{sequence} via the second rule. | |
5592 | Alternatively, nothing-at-all could be reduced into a @code{sequence} | |
5593 | via the first rule, and this could be combined with the @code{word} | |
5594 | using the third rule for @code{sequence}. | |
5595 | ||
5596 | There is also more than one way to reduce nothing-at-all into a | |
5597 | @code{sequence}. This can be done directly via the first rule, | |
5598 | or indirectly via @code{maybeword} and then the second rule. | |
5599 | ||
5600 | You might think that this is a distinction without a difference, because it | |
5601 | does not change whether any particular input is valid or not. But it does | |
5602 | affect which actions are run. One parsing order runs the second rule's | |
5603 | action; the other runs the first rule's action and the third rule's action. | |
5604 | In this example, the output of the program changes. | |
5605 | ||
5606 | Bison resolves a reduce/reduce conflict by choosing to use the rule that | |
5607 | appears first in the grammar, but it is very risky to rely on this. Every | |
5608 | reduce/reduce conflict must be studied and usually eliminated. Here is the | |
5609 | proper way to define @code{sequence}: | |
5610 | ||
5611 | @example | |
5612 | sequence: /* empty */ | |
5613 | @{ printf ("empty sequence\n"); @} | |
5614 | | sequence word | |
5615 | @{ printf ("added word %s\n", $2); @} | |
5616 | ; | |
5617 | @end example | |
5618 | ||
5619 | Here is another common error that yields a reduce/reduce conflict: | |
5620 | ||
5621 | @example | |
5622 | sequence: /* empty */ | |
5623 | | sequence words | |
5624 | | sequence redirects | |
5625 | ; | |
5626 | ||
5627 | words: /* empty */ | |
5628 | | words word | |
5629 | ; | |
5630 | ||
5631 | redirects:/* empty */ | |
5632 | | redirects redirect | |
5633 | ; | |
5634 | @end example | |
5635 | ||
5636 | @noindent | |
5637 | The intention here is to define a sequence which can contain either | |
5638 | @code{word} or @code{redirect} groupings. The individual definitions of | |
5639 | @code{sequence}, @code{words} and @code{redirects} are error-free, but the | |
5640 | three together make a subtle ambiguity: even an empty input can be parsed | |
5641 | in infinitely many ways! | |
5642 | ||
5643 | Consider: nothing-at-all could be a @code{words}. Or it could be two | |
5644 | @code{words} in a row, or three, or any number. It could equally well be a | |
5645 | @code{redirects}, or two, or any number. Or it could be a @code{words} | |
5646 | followed by three @code{redirects} and another @code{words}. And so on. | |
5647 | ||
5648 | Here are two ways to correct these rules. First, to make it a single level | |
5649 | of sequence: | |
5650 | ||
5651 | @example | |
5652 | sequence: /* empty */ | |
5653 | | sequence word | |
5654 | | sequence redirect | |
5655 | ; | |
5656 | @end example | |
5657 | ||
5658 | Second, to prevent either a @code{words} or a @code{redirects} | |
5659 | from being empty: | |
5660 | ||
5661 | @example | |
5662 | sequence: /* empty */ | |
5663 | | sequence words | |
5664 | | sequence redirects | |
5665 | ; | |
5666 | ||
5667 | words: word | |
5668 | | words word | |
5669 | ; | |
5670 | ||
5671 | redirects:redirect | |
5672 | | redirects redirect | |
5673 | ; | |
5674 | @end example | |
5675 | ||
342b8b6e | 5676 | @node Mystery Conflicts |
bfa74976 RS |
5677 | @section Mysterious Reduce/Reduce Conflicts |
5678 | ||
5679 | Sometimes reduce/reduce conflicts can occur that don't look warranted. | |
5680 | Here is an example: | |
5681 | ||
5682 | @example | |
5683 | @group | |
5684 | %token ID | |
5685 | ||
5686 | %% | |
5687 | def: param_spec return_spec ',' | |
5688 | ; | |
5689 | param_spec: | |
5690 | type | |
5691 | | name_list ':' type | |
5692 | ; | |
5693 | @end group | |
5694 | @group | |
5695 | return_spec: | |
5696 | type | |
5697 | | name ':' type | |
5698 | ; | |
5699 | @end group | |
5700 | @group | |
5701 | type: ID | |
5702 | ; | |
5703 | @end group | |
5704 | @group | |
5705 | name: ID | |
5706 | ; | |
5707 | name_list: | |
5708 | name | |
5709 | | name ',' name_list | |
5710 | ; | |
5711 | @end group | |
5712 | @end example | |
5713 | ||
5714 | It would seem that this grammar can be parsed with only a single token | |
742e4900 | 5715 | of lookahead: when a @code{param_spec} is being read, an @code{ID} is |
bfa74976 | 5716 | a @code{name} if a comma or colon follows, or a @code{type} if another |
c827f760 | 5717 | @code{ID} follows. In other words, this grammar is @acronym{LR}(1). |
bfa74976 | 5718 | |
c827f760 PE |
5719 | @cindex @acronym{LR}(1) |
5720 | @cindex @acronym{LALR}(1) | |
bfa74976 | 5721 | However, Bison, like most parser generators, cannot actually handle all |
c827f760 PE |
5722 | @acronym{LR}(1) grammars. In this grammar, two contexts, that after |
5723 | an @code{ID} | |
bfa74976 RS |
5724 | at the beginning of a @code{param_spec} and likewise at the beginning of |
5725 | a @code{return_spec}, are similar enough that Bison assumes they are the | |
5726 | same. They appear similar because the same set of rules would be | |
5727 | active---the rule for reducing to a @code{name} and that for reducing to | |
5728 | a @code{type}. Bison is unable to determine at that stage of processing | |
742e4900 | 5729 | that the rules would require different lookahead tokens in the two |
bfa74976 RS |
5730 | contexts, so it makes a single parser state for them both. Combining |
5731 | the two contexts causes a conflict later. In parser terminology, this | |
c827f760 | 5732 | occurrence means that the grammar is not @acronym{LALR}(1). |
bfa74976 RS |
5733 | |
5734 | In general, it is better to fix deficiencies than to document them. But | |
5735 | this particular deficiency is intrinsically hard to fix; parser | |
c827f760 PE |
5736 | generators that can handle @acronym{LR}(1) grammars are hard to write |
5737 | and tend to | |
bfa74976 RS |
5738 | produce parsers that are very large. In practice, Bison is more useful |
5739 | as it is now. | |
5740 | ||
5741 | When the problem arises, you can often fix it by identifying the two | |
a220f555 MA |
5742 | parser states that are being confused, and adding something to make them |
5743 | look distinct. In the above example, adding one rule to | |
bfa74976 RS |
5744 | @code{return_spec} as follows makes the problem go away: |
5745 | ||
5746 | @example | |
5747 | @group | |
5748 | %token BOGUS | |
5749 | @dots{} | |
5750 | %% | |
5751 | @dots{} | |
5752 | return_spec: | |
5753 | type | |
5754 | | name ':' type | |
5755 | /* This rule is never used. */ | |
5756 | | ID BOGUS | |
5757 | ; | |
5758 | @end group | |
5759 | @end example | |
5760 | ||
5761 | This corrects the problem because it introduces the possibility of an | |
5762 | additional active rule in the context after the @code{ID} at the beginning of | |
5763 | @code{return_spec}. This rule is not active in the corresponding context | |
5764 | in a @code{param_spec}, so the two contexts receive distinct parser states. | |
5765 | As long as the token @code{BOGUS} is never generated by @code{yylex}, | |
5766 | the added rule cannot alter the way actual input is parsed. | |
5767 | ||
5768 | In this particular example, there is another way to solve the problem: | |
5769 | rewrite the rule for @code{return_spec} to use @code{ID} directly | |
5770 | instead of via @code{name}. This also causes the two confusing | |
5771 | contexts to have different sets of active rules, because the one for | |
5772 | @code{return_spec} activates the altered rule for @code{return_spec} | |
5773 | rather than the one for @code{name}. | |
5774 | ||
5775 | @example | |
5776 | param_spec: | |
5777 | type | |
5778 | | name_list ':' type | |
5779 | ; | |
5780 | return_spec: | |
5781 | type | |
5782 | | ID ':' type | |
5783 | ; | |
5784 | @end example | |
5785 | ||
e054b190 PE |
5786 | For a more detailed exposition of @acronym{LALR}(1) parsers and parser |
5787 | generators, please see: | |
5788 | Frank DeRemer and Thomas Pennello, Efficient Computation of | |
5789 | @acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on | |
5790 | Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982), | |
5791 | pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}. | |
5792 | ||
fae437e8 | 5793 | @node Generalized LR Parsing |
c827f760 PE |
5794 | @section Generalized @acronym{LR} (@acronym{GLR}) Parsing |
5795 | @cindex @acronym{GLR} parsing | |
5796 | @cindex generalized @acronym{LR} (@acronym{GLR}) parsing | |
676385e2 | 5797 | @cindex ambiguous grammars |
9d9b8b70 | 5798 | @cindex nondeterministic parsing |
676385e2 | 5799 | |
fae437e8 AD |
5800 | Bison produces @emph{deterministic} parsers that choose uniquely |
5801 | when to reduce and which reduction to apply | |
742e4900 | 5802 | based on a summary of the preceding input and on one extra token of lookahead. |
676385e2 PH |
5803 | As a result, normal Bison handles a proper subset of the family of |
5804 | context-free languages. | |
fae437e8 | 5805 | Ambiguous grammars, since they have strings with more than one possible |
676385e2 PH |
5806 | sequence of reductions cannot have deterministic parsers in this sense. |
5807 | The same is true of languages that require more than one symbol of | |
742e4900 | 5808 | lookahead, since the parser lacks the information necessary to make a |
676385e2 | 5809 | decision at the point it must be made in a shift-reduce parser. |
fae437e8 | 5810 | Finally, as previously mentioned (@pxref{Mystery Conflicts}), |
676385e2 PH |
5811 | there are languages where Bison's particular choice of how to |
5812 | summarize the input seen so far loses necessary information. | |
5813 | ||
5814 | When you use the @samp{%glr-parser} declaration in your grammar file, | |
5815 | Bison generates a parser that uses a different algorithm, called | |
c827f760 PE |
5816 | Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR} |
5817 | parser uses the same basic | |
676385e2 PH |
5818 | algorithm for parsing as an ordinary Bison parser, but behaves |
5819 | differently in cases where there is a shift-reduce conflict that has not | |
fae437e8 | 5820 | been resolved by precedence rules (@pxref{Precedence}) or a |
c827f760 PE |
5821 | reduce-reduce conflict. When a @acronym{GLR} parser encounters such a |
5822 | situation, it | |
fae437e8 | 5823 | effectively @emph{splits} into a several parsers, one for each possible |
676385e2 PH |
5824 | shift or reduction. These parsers then proceed as usual, consuming |
5825 | tokens in lock-step. Some of the stacks may encounter other conflicts | |
fae437e8 | 5826 | and split further, with the result that instead of a sequence of states, |
c827f760 | 5827 | a Bison @acronym{GLR} parsing stack is what is in effect a tree of states. |
676385e2 PH |
5828 | |
5829 | In effect, each stack represents a guess as to what the proper parse | |
5830 | is. Additional input may indicate that a guess was wrong, in which case | |
5831 | the appropriate stack silently disappears. Otherwise, the semantics | |
fae437e8 | 5832 | actions generated in each stack are saved, rather than being executed |
676385e2 | 5833 | immediately. When a stack disappears, its saved semantic actions never |
fae437e8 | 5834 | get executed. When a reduction causes two stacks to become equivalent, |
676385e2 PH |
5835 | their sets of semantic actions are both saved with the state that |
5836 | results from the reduction. We say that two stacks are equivalent | |
fae437e8 | 5837 | when they both represent the same sequence of states, |
676385e2 PH |
5838 | and each pair of corresponding states represents a |
5839 | grammar symbol that produces the same segment of the input token | |
5840 | stream. | |
5841 | ||
5842 | Whenever the parser makes a transition from having multiple | |
c827f760 | 5843 | states to having one, it reverts to the normal @acronym{LALR}(1) parsing |
676385e2 PH |
5844 | algorithm, after resolving and executing the saved-up actions. |
5845 | At this transition, some of the states on the stack will have semantic | |
5846 | values that are sets (actually multisets) of possible actions. The | |
5847 | parser tries to pick one of the actions by first finding one whose rule | |
5848 | has the highest dynamic precedence, as set by the @samp{%dprec} | |
fae437e8 | 5849 | declaration. Otherwise, if the alternative actions are not ordered by |
676385e2 | 5850 | precedence, but there the same merging function is declared for both |
fae437e8 | 5851 | rules by the @samp{%merge} declaration, |
676385e2 PH |
5852 | Bison resolves and evaluates both and then calls the merge function on |
5853 | the result. Otherwise, it reports an ambiguity. | |
5854 | ||
c827f760 PE |
5855 | It is possible to use a data structure for the @acronym{GLR} parsing tree that |
5856 | permits the processing of any @acronym{LALR}(1) grammar in linear time (in the | |
5857 | size of the input), any unambiguous (not necessarily | |
5858 | @acronym{LALR}(1)) grammar in | |
fae437e8 | 5859 | quadratic worst-case time, and any general (possibly ambiguous) |
676385e2 PH |
5860 | context-free grammar in cubic worst-case time. However, Bison currently |
5861 | uses a simpler data structure that requires time proportional to the | |
5862 | length of the input times the maximum number of stacks required for any | |
9d9b8b70 | 5863 | prefix of the input. Thus, really ambiguous or nondeterministic |
676385e2 PH |
5864 | grammars can require exponential time and space to process. Such badly |
5865 | behaving examples, however, are not generally of practical interest. | |
9d9b8b70 | 5866 | Usually, nondeterminism in a grammar is local---the parser is ``in |
676385e2 | 5867 | doubt'' only for a few tokens at a time. Therefore, the current data |
c827f760 | 5868 | structure should generally be adequate. On @acronym{LALR}(1) portions of a |
676385e2 PH |
5869 | grammar, in particular, it is only slightly slower than with the default |
5870 | Bison parser. | |
5871 | ||
fa7e68c3 | 5872 | For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth |
f6481e2f PE |
5873 | Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style |
5874 | Generalised @acronym{LR} Parsers, Royal Holloway, University of | |
5875 | London, Department of Computer Science, TR-00-12, | |
5876 | @uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps}, | |
5877 | (2000-12-24). | |
5878 | ||
1a059451 PE |
5879 | @node Memory Management |
5880 | @section Memory Management, and How to Avoid Memory Exhaustion | |
5881 | @cindex memory exhaustion | |
5882 | @cindex memory management | |
bfa74976 RS |
5883 | @cindex stack overflow |
5884 | @cindex parser stack overflow | |
5885 | @cindex overflow of parser stack | |
5886 | ||
1a059451 | 5887 | The Bison parser stack can run out of memory if too many tokens are shifted and |
bfa74976 | 5888 | not reduced. When this happens, the parser function @code{yyparse} |
1a059451 | 5889 | calls @code{yyerror} and then returns 2. |
bfa74976 | 5890 | |
c827f760 | 5891 | Because Bison parsers have growing stacks, hitting the upper limit |
d1a1114f AD |
5892 | usually results from using a right recursion instead of a left |
5893 | recursion, @xref{Recursion, ,Recursive Rules}. | |
5894 | ||
bfa74976 RS |
5895 | @vindex YYMAXDEPTH |
5896 | By defining the macro @code{YYMAXDEPTH}, you can control how deep the | |
1a059451 | 5897 | parser stack can become before memory is exhausted. Define the |
bfa74976 RS |
5898 | macro with a value that is an integer. This value is the maximum number |
5899 | of tokens that can be shifted (and not reduced) before overflow. | |
bfa74976 RS |
5900 | |
5901 | The stack space allowed is not necessarily allocated. If you specify a | |
1a059451 | 5902 | large value for @code{YYMAXDEPTH}, the parser normally allocates a small |
bfa74976 RS |
5903 | stack at first, and then makes it bigger by stages as needed. This |
5904 | increasing allocation happens automatically and silently. Therefore, | |
5905 | you do not need to make @code{YYMAXDEPTH} painfully small merely to save | |
5906 | space for ordinary inputs that do not need much stack. | |
5907 | ||
d7e14fc0 PE |
5908 | However, do not allow @code{YYMAXDEPTH} to be a value so large that |
5909 | arithmetic overflow could occur when calculating the size of the stack | |
5910 | space. Also, do not allow @code{YYMAXDEPTH} to be less than | |
5911 | @code{YYINITDEPTH}. | |
5912 | ||
bfa74976 RS |
5913 | @cindex default stack limit |
5914 | The default value of @code{YYMAXDEPTH}, if you do not define it, is | |
5915 | 10000. | |
5916 | ||
5917 | @vindex YYINITDEPTH | |
5918 | You can control how much stack is allocated initially by defining the | |
d7e14fc0 PE |
5919 | macro @code{YYINITDEPTH} to a positive integer. For the C |
5920 | @acronym{LALR}(1) parser, this value must be a compile-time constant | |
5921 | unless you are assuming C99 or some other target language or compiler | |
5922 | that allows variable-length arrays. The default is 200. | |
5923 | ||
1a059451 | 5924 | Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}. |
bfa74976 | 5925 | |
d1a1114f | 5926 | @c FIXME: C++ output. |
c827f760 | 5927 | Because of semantical differences between C and C++, the |
1a059451 PE |
5928 | @acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled |
5929 | by C++ compilers. In this precise case (compiling a C parser as C++) you are | |
5930 | suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix | |
5931 | this deficiency in a future release. | |
d1a1114f | 5932 | |
342b8b6e | 5933 | @node Error Recovery |
bfa74976 RS |
5934 | @chapter Error Recovery |
5935 | @cindex error recovery | |
5936 | @cindex recovery from errors | |
5937 | ||
6e649e65 | 5938 | It is not usually acceptable to have a program terminate on a syntax |
bfa74976 RS |
5939 | error. For example, a compiler should recover sufficiently to parse the |
5940 | rest of the input file and check it for errors; a calculator should accept | |
5941 | another expression. | |
5942 | ||
5943 | In a simple interactive command parser where each input is one line, it may | |
5944 | be sufficient to allow @code{yyparse} to return 1 on error and have the | |
5945 | caller ignore the rest of the input line when that happens (and then call | |
5946 | @code{yyparse} again). But this is inadequate for a compiler, because it | |
5947 | forgets all the syntactic context leading up to the error. A syntax error | |
5948 | deep within a function in the compiler input should not cause the compiler | |
5949 | to treat the following line like the beginning of a source file. | |
5950 | ||
5951 | @findex error | |
5952 | You can define how to recover from a syntax error by writing rules to | |
5953 | recognize the special token @code{error}. This is a terminal symbol that | |
5954 | is always defined (you need not declare it) and reserved for error | |
5955 | handling. The Bison parser generates an @code{error} token whenever a | |
5956 | syntax error happens; if you have provided a rule to recognize this token | |
13863333 | 5957 | in the current context, the parse can continue. |
bfa74976 RS |
5958 | |
5959 | For example: | |
5960 | ||
5961 | @example | |
5962 | stmnts: /* empty string */ | |
5963 | | stmnts '\n' | |
5964 | | stmnts exp '\n' | |
5965 | | stmnts error '\n' | |
5966 | @end example | |
5967 | ||
5968 | The fourth rule in this example says that an error followed by a newline | |
5969 | makes a valid addition to any @code{stmnts}. | |
5970 | ||
5971 | What happens if a syntax error occurs in the middle of an @code{exp}? The | |
5972 | error recovery rule, interpreted strictly, applies to the precise sequence | |
5973 | of a @code{stmnts}, an @code{error} and a newline. If an error occurs in | |
5974 | the middle of an @code{exp}, there will probably be some additional tokens | |
5975 | and subexpressions on the stack after the last @code{stmnts}, and there | |
5976 | will be tokens to read before the next newline. So the rule is not | |
5977 | applicable in the ordinary way. | |
5978 | ||
5979 | But Bison can force the situation to fit the rule, by discarding part of | |
72f889cc AD |
5980 | the semantic context and part of the input. First it discards states |
5981 | and objects from the stack until it gets back to a state in which the | |
bfa74976 | 5982 | @code{error} token is acceptable. (This means that the subexpressions |
72f889cc AD |
5983 | already parsed are discarded, back to the last complete @code{stmnts}.) |
5984 | At this point the @code{error} token can be shifted. Then, if the old | |
742e4900 | 5985 | lookahead token is not acceptable to be shifted next, the parser reads |
bfa74976 | 5986 | tokens and discards them until it finds a token which is acceptable. In |
72f889cc AD |
5987 | this example, Bison reads and discards input until the next newline so |
5988 | that the fourth rule can apply. Note that discarded symbols are | |
5989 | possible sources of memory leaks, see @ref{Destructor Decl, , Freeing | |
5990 | Discarded Symbols}, for a means to reclaim this memory. | |
bfa74976 RS |
5991 | |
5992 | The choice of error rules in the grammar is a choice of strategies for | |
5993 | error recovery. A simple and useful strategy is simply to skip the rest of | |
5994 | the current input line or current statement if an error is detected: | |
5995 | ||
5996 | @example | |
72d2299c | 5997 | stmnt: error ';' /* On error, skip until ';' is read. */ |
bfa74976 RS |
5998 | @end example |
5999 | ||
6000 | It is also useful to recover to the matching close-delimiter of an | |
6001 | opening-delimiter that has already been parsed. Otherwise the | |
6002 | close-delimiter will probably appear to be unmatched, and generate another, | |
6003 | spurious error message: | |
6004 | ||
6005 | @example | |
6006 | primary: '(' expr ')' | |
6007 | | '(' error ')' | |
6008 | @dots{} | |
6009 | ; | |
6010 | @end example | |
6011 | ||
6012 | Error recovery strategies are necessarily guesses. When they guess wrong, | |
6013 | one syntax error often leads to another. In the above example, the error | |
6014 | recovery rule guesses that an error is due to bad input within one | |
6015 | @code{stmnt}. Suppose that instead a spurious semicolon is inserted in the | |
6016 | middle of a valid @code{stmnt}. After the error recovery rule recovers | |
6017 | from the first error, another syntax error will be found straightaway, | |
6018 | since the text following the spurious semicolon is also an invalid | |
6019 | @code{stmnt}. | |
6020 | ||
6021 | To prevent an outpouring of error messages, the parser will output no error | |
6022 | message for another syntax error that happens shortly after the first; only | |
6023 | after three consecutive input tokens have been successfully shifted will | |
6024 | error messages resume. | |
6025 | ||
6026 | Note that rules which accept the @code{error} token may have actions, just | |
6027 | as any other rules can. | |
6028 | ||
6029 | @findex yyerrok | |
6030 | You can make error messages resume immediately by using the macro | |
6031 | @code{yyerrok} in an action. If you do this in the error rule's action, no | |
6032 | error messages will be suppressed. This macro requires no arguments; | |
6033 | @samp{yyerrok;} is a valid C statement. | |
6034 | ||
6035 | @findex yyclearin | |
742e4900 | 6036 | The previous lookahead token is reanalyzed immediately after an error. If |
bfa74976 RS |
6037 | this is unacceptable, then the macro @code{yyclearin} may be used to clear |
6038 | this token. Write the statement @samp{yyclearin;} in the error rule's | |
6039 | action. | |
32c29292 | 6040 | @xref{Action Features, ,Special Features for Use in Actions}. |
bfa74976 | 6041 | |
6e649e65 | 6042 | For example, suppose that on a syntax error, an error handling routine is |
bfa74976 RS |
6043 | called that advances the input stream to some point where parsing should |
6044 | once again commence. The next symbol returned by the lexical scanner is | |
742e4900 | 6045 | probably correct. The previous lookahead token ought to be discarded |
bfa74976 RS |
6046 | with @samp{yyclearin;}. |
6047 | ||
6048 | @vindex YYRECOVERING | |
02103984 PE |
6049 | The expression @code{YYRECOVERING ()} yields 1 when the parser |
6050 | is recovering from a syntax error, and 0 otherwise. | |
6051 | Syntax error diagnostics are suppressed while recovering from a syntax | |
6052 | error. | |
bfa74976 | 6053 | |
342b8b6e | 6054 | @node Context Dependency |
bfa74976 RS |
6055 | @chapter Handling Context Dependencies |
6056 | ||
6057 | The Bison paradigm is to parse tokens first, then group them into larger | |
6058 | syntactic units. In many languages, the meaning of a token is affected by | |
6059 | its context. Although this violates the Bison paradigm, certain techniques | |
6060 | (known as @dfn{kludges}) may enable you to write Bison parsers for such | |
6061 | languages. | |
6062 | ||
6063 | @menu | |
6064 | * Semantic Tokens:: Token parsing can depend on the semantic context. | |
6065 | * Lexical Tie-ins:: Token parsing can depend on the syntactic context. | |
6066 | * Tie-in Recovery:: Lexical tie-ins have implications for how | |
6067 | error recovery rules must be written. | |
6068 | @end menu | |
6069 | ||
6070 | (Actually, ``kludge'' means any technique that gets its job done but is | |
6071 | neither clean nor robust.) | |
6072 | ||
342b8b6e | 6073 | @node Semantic Tokens |
bfa74976 RS |
6074 | @section Semantic Info in Token Types |
6075 | ||
6076 | The C language has a context dependency: the way an identifier is used | |
6077 | depends on what its current meaning is. For example, consider this: | |
6078 | ||
6079 | @example | |
6080 | foo (x); | |
6081 | @end example | |
6082 | ||
6083 | This looks like a function call statement, but if @code{foo} is a typedef | |
6084 | name, then this is actually a declaration of @code{x}. How can a Bison | |
6085 | parser for C decide how to parse this input? | |
6086 | ||
c827f760 | 6087 | The method used in @acronym{GNU} C is to have two different token types, |
bfa74976 RS |
6088 | @code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an |
6089 | identifier, it looks up the current declaration of the identifier in order | |
6090 | to decide which token type to return: @code{TYPENAME} if the identifier is | |
6091 | declared as a typedef, @code{IDENTIFIER} otherwise. | |
6092 | ||
6093 | The grammar rules can then express the context dependency by the choice of | |
6094 | token type to recognize. @code{IDENTIFIER} is accepted as an expression, | |
6095 | but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but | |
6096 | @code{IDENTIFIER} cannot. In contexts where the meaning of the identifier | |
6097 | is @emph{not} significant, such as in declarations that can shadow a | |
6098 | typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is | |
6099 | accepted---there is one rule for each of the two token types. | |
6100 | ||
6101 | This technique is simple to use if the decision of which kinds of | |
6102 | identifiers to allow is made at a place close to where the identifier is | |
6103 | parsed. But in C this is not always so: C allows a declaration to | |
6104 | redeclare a typedef name provided an explicit type has been specified | |
6105 | earlier: | |
6106 | ||
6107 | @example | |
3a4f411f PE |
6108 | typedef int foo, bar; |
6109 | int baz (void) | |
6110 | @{ | |
6111 | static bar (bar); /* @r{redeclare @code{bar} as static variable} */ | |
6112 | extern foo foo (foo); /* @r{redeclare @code{foo} as function} */ | |
6113 | return foo (bar); | |
6114 | @} | |
bfa74976 RS |
6115 | @end example |
6116 | ||
6117 | Unfortunately, the name being declared is separated from the declaration | |
6118 | construct itself by a complicated syntactic structure---the ``declarator''. | |
6119 | ||
9ecbd125 | 6120 | As a result, part of the Bison parser for C needs to be duplicated, with |
14ded682 AD |
6121 | all the nonterminal names changed: once for parsing a declaration in |
6122 | which a typedef name can be redefined, and once for parsing a | |
6123 | declaration in which that can't be done. Here is a part of the | |
6124 | duplication, with actions omitted for brevity: | |
bfa74976 RS |
6125 | |
6126 | @example | |
6127 | initdcl: | |
6128 | declarator maybeasm '=' | |
6129 | init | |
6130 | | declarator maybeasm | |
6131 | ; | |
6132 | ||
6133 | notype_initdcl: | |
6134 | notype_declarator maybeasm '=' | |
6135 | init | |
6136 | | notype_declarator maybeasm | |
6137 | ; | |
6138 | @end example | |
6139 | ||
6140 | @noindent | |
6141 | Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl} | |
6142 | cannot. The distinction between @code{declarator} and | |
6143 | @code{notype_declarator} is the same sort of thing. | |
6144 | ||
6145 | There is some similarity between this technique and a lexical tie-in | |
6146 | (described next), in that information which alters the lexical analysis is | |
6147 | changed during parsing by other parts of the program. The difference is | |
6148 | here the information is global, and is used for other purposes in the | |
6149 | program. A true lexical tie-in has a special-purpose flag controlled by | |
6150 | the syntactic context. | |
6151 | ||
342b8b6e | 6152 | @node Lexical Tie-ins |
bfa74976 RS |
6153 | @section Lexical Tie-ins |
6154 | @cindex lexical tie-in | |
6155 | ||
6156 | One way to handle context-dependency is the @dfn{lexical tie-in}: a flag | |
6157 | which is set by Bison actions, whose purpose is to alter the way tokens are | |
6158 | parsed. | |
6159 | ||
6160 | For example, suppose we have a language vaguely like C, but with a special | |
6161 | construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes | |
6162 | an expression in parentheses in which all integers are hexadecimal. In | |
6163 | particular, the token @samp{a1b} must be treated as an integer rather than | |
6164 | as an identifier if it appears in that context. Here is how you can do it: | |
6165 | ||
6166 | @example | |
6167 | @group | |
6168 | %@{ | |
38a92d50 PE |
6169 | int hexflag; |
6170 | int yylex (void); | |
6171 | void yyerror (char const *); | |
bfa74976 RS |
6172 | %@} |
6173 | %% | |
6174 | @dots{} | |
6175 | @end group | |
6176 | @group | |
6177 | expr: IDENTIFIER | |
6178 | | constant | |
6179 | | HEX '(' | |
6180 | @{ hexflag = 1; @} | |
6181 | expr ')' | |
6182 | @{ hexflag = 0; | |
6183 | $$ = $4; @} | |
6184 | | expr '+' expr | |
6185 | @{ $$ = make_sum ($1, $3); @} | |
6186 | @dots{} | |
6187 | ; | |
6188 | @end group | |
6189 | ||
6190 | @group | |
6191 | constant: | |
6192 | INTEGER | |
6193 | | STRING | |
6194 | ; | |
6195 | @end group | |
6196 | @end example | |
6197 | ||
6198 | @noindent | |
6199 | Here we assume that @code{yylex} looks at the value of @code{hexflag}; when | |
6200 | it is nonzero, all integers are parsed in hexadecimal, and tokens starting | |
6201 | with letters are parsed as integers if possible. | |
6202 | ||
342b8b6e AD |
6203 | The declaration of @code{hexflag} shown in the prologue of the parser file |
6204 | is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}). | |
75f5aaea | 6205 | You must also write the code in @code{yylex} to obey the flag. |
bfa74976 | 6206 | |
342b8b6e | 6207 | @node Tie-in Recovery |
bfa74976 RS |
6208 | @section Lexical Tie-ins and Error Recovery |
6209 | ||
6210 | Lexical tie-ins make strict demands on any error recovery rules you have. | |
6211 | @xref{Error Recovery}. | |
6212 | ||
6213 | The reason for this is that the purpose of an error recovery rule is to | |
6214 | abort the parsing of one construct and resume in some larger construct. | |
6215 | For example, in C-like languages, a typical error recovery rule is to skip | |
6216 | tokens until the next semicolon, and then start a new statement, like this: | |
6217 | ||
6218 | @example | |
6219 | stmt: expr ';' | |
6220 | | IF '(' expr ')' stmt @{ @dots{} @} | |
6221 | @dots{} | |
6222 | error ';' | |
6223 | @{ hexflag = 0; @} | |
6224 | ; | |
6225 | @end example | |
6226 | ||
6227 | If there is a syntax error in the middle of a @samp{hex (@var{expr})} | |
6228 | construct, this error rule will apply, and then the action for the | |
6229 | completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would | |
6230 | remain set for the entire rest of the input, or until the next @code{hex} | |
6231 | keyword, causing identifiers to be misinterpreted as integers. | |
6232 | ||
6233 | To avoid this problem the error recovery rule itself clears @code{hexflag}. | |
6234 | ||
6235 | There may also be an error recovery rule that works within expressions. | |
6236 | For example, there could be a rule which applies within parentheses | |
6237 | and skips to the close-parenthesis: | |
6238 | ||
6239 | @example | |
6240 | @group | |
6241 | expr: @dots{} | |
6242 | | '(' expr ')' | |
6243 | @{ $$ = $2; @} | |
6244 | | '(' error ')' | |
6245 | @dots{} | |
6246 | @end group | |
6247 | @end example | |
6248 | ||
6249 | If this rule acts within the @code{hex} construct, it is not going to abort | |
6250 | that construct (since it applies to an inner level of parentheses within | |
6251 | the construct). Therefore, it should not clear the flag: the rest of | |
6252 | the @code{hex} construct should be parsed with the flag still in effect. | |
6253 | ||
6254 | What if there is an error recovery rule which might abort out of the | |
6255 | @code{hex} construct or might not, depending on circumstances? There is no | |
6256 | way you can write the action to determine whether a @code{hex} construct is | |
6257 | being aborted or not. So if you are using a lexical tie-in, you had better | |
6258 | make sure your error recovery rules are not of this kind. Each rule must | |
6259 | be such that you can be sure that it always will, or always won't, have to | |
6260 | clear the flag. | |
6261 | ||
ec3bc396 AD |
6262 | @c ================================================== Debugging Your Parser |
6263 | ||
342b8b6e | 6264 | @node Debugging |
bfa74976 | 6265 | @chapter Debugging Your Parser |
ec3bc396 AD |
6266 | |
6267 | Developing a parser can be a challenge, especially if you don't | |
6268 | understand the algorithm (@pxref{Algorithm, ,The Bison Parser | |
6269 | Algorithm}). Even so, sometimes a detailed description of the automaton | |
6270 | can help (@pxref{Understanding, , Understanding Your Parser}), or | |
6271 | tracing the execution of the parser can give some insight on why it | |
6272 | behaves improperly (@pxref{Tracing, , Tracing Your Parser}). | |
6273 | ||
6274 | @menu | |
6275 | * Understanding:: Understanding the structure of your parser. | |
6276 | * Tracing:: Tracing the execution of your parser. | |
6277 | @end menu | |
6278 | ||
6279 | @node Understanding | |
6280 | @section Understanding Your Parser | |
6281 | ||
6282 | As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm}) | |
6283 | Bison parsers are @dfn{shift/reduce automata}. In some cases (much more | |
6284 | frequent than one would hope), looking at this automaton is required to | |
6285 | tune or simply fix a parser. Bison provides two different | |
c827f760 | 6286 | representation of it, either textually or graphically (as a @acronym{VCG} |
ec3bc396 AD |
6287 | file). |
6288 | ||
6289 | The textual file is generated when the options @option{--report} or | |
6290 | @option{--verbose} are specified, see @xref{Invocation, , Invoking | |
6291 | Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from | |
6292 | the parser output file name, and adding @samp{.output} instead. | |
6293 | Therefore, if the input file is @file{foo.y}, then the parser file is | |
6294 | called @file{foo.tab.c} by default. As a consequence, the verbose | |
6295 | output file is called @file{foo.output}. | |
6296 | ||
6297 | The following grammar file, @file{calc.y}, will be used in the sequel: | |
6298 | ||
6299 | @example | |
6300 | %token NUM STR | |
6301 | %left '+' '-' | |
6302 | %left '*' | |
6303 | %% | |
6304 | exp: exp '+' exp | |
6305 | | exp '-' exp | |
6306 | | exp '*' exp | |
6307 | | exp '/' exp | |
6308 | | NUM | |
6309 | ; | |
6310 | useless: STR; | |
6311 | %% | |
6312 | @end example | |
6313 | ||
88bce5a2 AD |
6314 | @command{bison} reports: |
6315 | ||
6316 | @example | |
6317 | calc.y: warning: 1 useless nonterminal and 1 useless rule | |
6318 | calc.y:11.1-7: warning: useless nonterminal: useless | |
5a99098d PE |
6319 | calc.y:11.10-12: warning: useless rule: useless: STR |
6320 | calc.y: conflicts: 7 shift/reduce | |
88bce5a2 AD |
6321 | @end example |
6322 | ||
6323 | When given @option{--report=state}, in addition to @file{calc.tab.c}, it | |
6324 | creates a file @file{calc.output} with contents detailed below. The | |
6325 | order of the output and the exact presentation might vary, but the | |
6326 | interpretation is the same. | |
ec3bc396 AD |
6327 | |
6328 | The first section includes details on conflicts that were solved thanks | |
6329 | to precedence and/or associativity: | |
6330 | ||
6331 | @example | |
6332 | Conflict in state 8 between rule 2 and token '+' resolved as reduce. | |
6333 | Conflict in state 8 between rule 2 and token '-' resolved as reduce. | |
6334 | Conflict in state 8 between rule 2 and token '*' resolved as shift. | |
6335 | @exdent @dots{} | |
6336 | @end example | |
6337 | ||
6338 | @noindent | |
6339 | The next section lists states that still have conflicts. | |
6340 | ||
6341 | @example | |
5a99098d PE |
6342 | State 8 conflicts: 1 shift/reduce |
6343 | State 9 conflicts: 1 shift/reduce | |
6344 | State 10 conflicts: 1 shift/reduce | |
6345 | State 11 conflicts: 4 shift/reduce | |
ec3bc396 AD |
6346 | @end example |
6347 | ||
6348 | @noindent | |
6349 | @cindex token, useless | |
6350 | @cindex useless token | |
6351 | @cindex nonterminal, useless | |
6352 | @cindex useless nonterminal | |
6353 | @cindex rule, useless | |
6354 | @cindex useless rule | |
6355 | The next section reports useless tokens, nonterminal and rules. Useless | |
6356 | nonterminals and rules are removed in order to produce a smaller parser, | |
6357 | but useless tokens are preserved, since they might be used by the | |
6358 | scanner (note the difference between ``useless'' and ``not used'' | |
6359 | below): | |
6360 | ||
6361 | @example | |
6362 | Useless nonterminals: | |
6363 | useless | |
6364 | ||
6365 | Terminals which are not used: | |
6366 | STR | |
6367 | ||
6368 | Useless rules: | |
6369 | #6 useless: STR; | |
6370 | @end example | |
6371 | ||
6372 | @noindent | |
6373 | The next section reproduces the exact grammar that Bison used: | |
6374 | ||
6375 | @example | |
6376 | Grammar | |
6377 | ||
6378 | Number, Line, Rule | |
88bce5a2 | 6379 | 0 5 $accept -> exp $end |
ec3bc396 AD |
6380 | 1 5 exp -> exp '+' exp |
6381 | 2 6 exp -> exp '-' exp | |
6382 | 3 7 exp -> exp '*' exp | |
6383 | 4 8 exp -> exp '/' exp | |
6384 | 5 9 exp -> NUM | |
6385 | @end example | |
6386 | ||
6387 | @noindent | |
6388 | and reports the uses of the symbols: | |
6389 | ||
6390 | @example | |
6391 | Terminals, with rules where they appear | |
6392 | ||
88bce5a2 | 6393 | $end (0) 0 |
ec3bc396 AD |
6394 | '*' (42) 3 |
6395 | '+' (43) 1 | |
6396 | '-' (45) 2 | |
6397 | '/' (47) 4 | |
6398 | error (256) | |
6399 | NUM (258) 5 | |
6400 | ||
6401 | Nonterminals, with rules where they appear | |
6402 | ||
88bce5a2 | 6403 | $accept (8) |
ec3bc396 AD |
6404 | on left: 0 |
6405 | exp (9) | |
6406 | on left: 1 2 3 4 5, on right: 0 1 2 3 4 | |
6407 | @end example | |
6408 | ||
6409 | @noindent | |
6410 | @cindex item | |
6411 | @cindex pointed rule | |
6412 | @cindex rule, pointed | |
6413 | Bison then proceeds onto the automaton itself, describing each state | |
6414 | with it set of @dfn{items}, also known as @dfn{pointed rules}. Each | |
6415 | item is a production rule together with a point (marked by @samp{.}) | |
6416 | that the input cursor. | |
6417 | ||
6418 | @example | |
6419 | state 0 | |
6420 | ||
88bce5a2 | 6421 | $accept -> . exp $ (rule 0) |
ec3bc396 | 6422 | |
2a8d363a | 6423 | NUM shift, and go to state 1 |
ec3bc396 | 6424 | |
2a8d363a | 6425 | exp go to state 2 |
ec3bc396 AD |
6426 | @end example |
6427 | ||
6428 | This reads as follows: ``state 0 corresponds to being at the very | |
6429 | beginning of the parsing, in the initial rule, right before the start | |
6430 | symbol (here, @code{exp}). When the parser returns to this state right | |
6431 | after having reduced a rule that produced an @code{exp}, the control | |
6432 | flow jumps to state 2. If there is no such transition on a nonterminal | |
742e4900 | 6433 | symbol, and the lookahead is a @code{NUM}, then this token is shifted on |
ec3bc396 | 6434 | the parse stack, and the control flow jumps to state 1. Any other |
742e4900 | 6435 | lookahead triggers a syntax error.'' |
ec3bc396 AD |
6436 | |
6437 | @cindex core, item set | |
6438 | @cindex item set core | |
6439 | @cindex kernel, item set | |
6440 | @cindex item set core | |
6441 | Even though the only active rule in state 0 seems to be rule 0, the | |
742e4900 | 6442 | report lists @code{NUM} as a lookahead token because @code{NUM} can be |
ec3bc396 AD |
6443 | at the beginning of any rule deriving an @code{exp}. By default Bison |
6444 | reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if | |
6445 | you want to see more detail you can invoke @command{bison} with | |
6446 | @option{--report=itemset} to list all the items, include those that can | |
6447 | be derived: | |
6448 | ||
6449 | @example | |
6450 | state 0 | |
6451 | ||
88bce5a2 | 6452 | $accept -> . exp $ (rule 0) |
ec3bc396 AD |
6453 | exp -> . exp '+' exp (rule 1) |
6454 | exp -> . exp '-' exp (rule 2) | |
6455 | exp -> . exp '*' exp (rule 3) | |
6456 | exp -> . exp '/' exp (rule 4) | |
6457 | exp -> . NUM (rule 5) | |
6458 | ||
6459 | NUM shift, and go to state 1 | |
6460 | ||
6461 | exp go to state 2 | |
6462 | @end example | |
6463 | ||
6464 | @noindent | |
6465 | In the state 1... | |
6466 | ||
6467 | @example | |
6468 | state 1 | |
6469 | ||
6470 | exp -> NUM . (rule 5) | |
6471 | ||
2a8d363a | 6472 | $default reduce using rule 5 (exp) |
ec3bc396 AD |
6473 | @end example |
6474 | ||
6475 | @noindent | |
742e4900 | 6476 | the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token |
ec3bc396 AD |
6477 | (@samp{$default}), the parser will reduce it. If it was coming from |
6478 | state 0, then, after this reduction it will return to state 0, and will | |
6479 | jump to state 2 (@samp{exp: go to state 2}). | |
6480 | ||
6481 | @example | |
6482 | state 2 | |
6483 | ||
88bce5a2 | 6484 | $accept -> exp . $ (rule 0) |
ec3bc396 AD |
6485 | exp -> exp . '+' exp (rule 1) |
6486 | exp -> exp . '-' exp (rule 2) | |
6487 | exp -> exp . '*' exp (rule 3) | |
6488 | exp -> exp . '/' exp (rule 4) | |
6489 | ||
2a8d363a AD |
6490 | $ shift, and go to state 3 |
6491 | '+' shift, and go to state 4 | |
6492 | '-' shift, and go to state 5 | |
6493 | '*' shift, and go to state 6 | |
6494 | '/' shift, and go to state 7 | |
ec3bc396 AD |
6495 | @end example |
6496 | ||
6497 | @noindent | |
6498 | In state 2, the automaton can only shift a symbol. For instance, | |
742e4900 | 6499 | because of the item @samp{exp -> exp . '+' exp}, if the lookahead if |
ec3bc396 AD |
6500 | @samp{+}, it will be shifted on the parse stack, and the automaton |
6501 | control will jump to state 4, corresponding to the item @samp{exp -> exp | |
6502 | '+' . exp}. Since there is no default action, any other token than | |
6e649e65 | 6503 | those listed above will trigger a syntax error. |
ec3bc396 AD |
6504 | |
6505 | The state 3 is named the @dfn{final state}, or the @dfn{accepting | |
6506 | state}: | |
6507 | ||
6508 | @example | |
6509 | state 3 | |
6510 | ||
88bce5a2 | 6511 | $accept -> exp $ . (rule 0) |
ec3bc396 | 6512 | |
2a8d363a | 6513 | $default accept |
ec3bc396 AD |
6514 | @end example |
6515 | ||
6516 | @noindent | |
6517 | the initial rule is completed (the start symbol and the end | |
6518 | of input were read), the parsing exits successfully. | |
6519 | ||
6520 | The interpretation of states 4 to 7 is straightforward, and is left to | |
6521 | the reader. | |
6522 | ||
6523 | @example | |
6524 | state 4 | |
6525 | ||
6526 | exp -> exp '+' . exp (rule 1) | |
6527 | ||
2a8d363a | 6528 | NUM shift, and go to state 1 |
ec3bc396 | 6529 | |
2a8d363a | 6530 | exp go to state 8 |
ec3bc396 AD |
6531 | |
6532 | state 5 | |
6533 | ||
6534 | exp -> exp '-' . exp (rule 2) | |
6535 | ||
2a8d363a | 6536 | NUM shift, and go to state 1 |
ec3bc396 | 6537 | |
2a8d363a | 6538 | exp go to state 9 |
ec3bc396 AD |
6539 | |
6540 | state 6 | |
6541 | ||
6542 | exp -> exp '*' . exp (rule 3) | |
6543 | ||
2a8d363a | 6544 | NUM shift, and go to state 1 |
ec3bc396 | 6545 | |
2a8d363a | 6546 | exp go to state 10 |
ec3bc396 AD |
6547 | |
6548 | state 7 | |
6549 | ||
6550 | exp -> exp '/' . exp (rule 4) | |
6551 | ||
2a8d363a | 6552 | NUM shift, and go to state 1 |
ec3bc396 | 6553 | |
2a8d363a | 6554 | exp go to state 11 |
ec3bc396 AD |
6555 | @end example |
6556 | ||
5a99098d PE |
6557 | As was announced in beginning of the report, @samp{State 8 conflicts: |
6558 | 1 shift/reduce}: | |
ec3bc396 AD |
6559 | |
6560 | @example | |
6561 | state 8 | |
6562 | ||
6563 | exp -> exp . '+' exp (rule 1) | |
6564 | exp -> exp '+' exp . (rule 1) | |
6565 | exp -> exp . '-' exp (rule 2) | |
6566 | exp -> exp . '*' exp (rule 3) | |
6567 | exp -> exp . '/' exp (rule 4) | |
6568 | ||
2a8d363a AD |
6569 | '*' shift, and go to state 6 |
6570 | '/' shift, and go to state 7 | |
ec3bc396 | 6571 | |
2a8d363a AD |
6572 | '/' [reduce using rule 1 (exp)] |
6573 | $default reduce using rule 1 (exp) | |
ec3bc396 AD |
6574 | @end example |
6575 | ||
742e4900 | 6576 | Indeed, there are two actions associated to the lookahead @samp{/}: |
ec3bc396 AD |
6577 | either shifting (and going to state 7), or reducing rule 1. The |
6578 | conflict means that either the grammar is ambiguous, or the parser lacks | |
6579 | information to make the right decision. Indeed the grammar is | |
6580 | ambiguous, as, since we did not specify the precedence of @samp{/}, the | |
6581 | sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM / | |
6582 | NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) / | |
6583 | NUM}, which corresponds to reducing rule 1. | |
6584 | ||
c827f760 | 6585 | Because in @acronym{LALR}(1) parsing a single decision can be made, Bison |
ec3bc396 AD |
6586 | arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, , |
6587 | Shift/Reduce Conflicts}. Discarded actions are reported in between | |
6588 | square brackets. | |
6589 | ||
6590 | Note that all the previous states had a single possible action: either | |
6591 | shifting the next token and going to the corresponding state, or | |
6592 | reducing a single rule. In the other cases, i.e., when shifting | |
6593 | @emph{and} reducing is possible or when @emph{several} reductions are | |
742e4900 JD |
6594 | possible, the lookahead is required to select the action. State 8 is |
6595 | one such state: if the lookahead is @samp{*} or @samp{/} then the action | |
ec3bc396 AD |
6596 | is shifting, otherwise the action is reducing rule 1. In other words, |
6597 | the first two items, corresponding to rule 1, are not eligible when the | |
742e4900 | 6598 | lookahead token is @samp{*}, since we specified that @samp{*} has higher |
8dd162d3 | 6599 | precedence than @samp{+}. More generally, some items are eligible only |
742e4900 JD |
6600 | with some set of possible lookahead tokens. When run with |
6601 | @option{--report=lookahead}, Bison specifies these lookahead tokens: | |
ec3bc396 AD |
6602 | |
6603 | @example | |
6604 | state 8 | |
6605 | ||
6606 | exp -> exp . '+' exp [$, '+', '-', '/'] (rule 1) | |
6607 | exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1) | |
6608 | exp -> exp . '-' exp (rule 2) | |
6609 | exp -> exp . '*' exp (rule 3) | |
6610 | exp -> exp . '/' exp (rule 4) | |
6611 | ||
6612 | '*' shift, and go to state 6 | |
6613 | '/' shift, and go to state 7 | |
6614 | ||
6615 | '/' [reduce using rule 1 (exp)] | |
6616 | $default reduce using rule 1 (exp) | |
6617 | @end example | |
6618 | ||
6619 | The remaining states are similar: | |
6620 | ||
6621 | @example | |
6622 | state 9 | |
6623 | ||
6624 | exp -> exp . '+' exp (rule 1) | |
6625 | exp -> exp . '-' exp (rule 2) | |
6626 | exp -> exp '-' exp . (rule 2) | |
6627 | exp -> exp . '*' exp (rule 3) | |
6628 | exp -> exp . '/' exp (rule 4) | |
6629 | ||
2a8d363a AD |
6630 | '*' shift, and go to state 6 |
6631 | '/' shift, and go to state 7 | |
ec3bc396 | 6632 | |
2a8d363a AD |
6633 | '/' [reduce using rule 2 (exp)] |
6634 | $default reduce using rule 2 (exp) | |
ec3bc396 AD |
6635 | |
6636 | state 10 | |
6637 | ||
6638 | exp -> exp . '+' exp (rule 1) | |
6639 | exp -> exp . '-' exp (rule 2) | |
6640 | exp -> exp . '*' exp (rule 3) | |
6641 | exp -> exp '*' exp . (rule 3) | |
6642 | exp -> exp . '/' exp (rule 4) | |
6643 | ||
2a8d363a | 6644 | '/' shift, and go to state 7 |
ec3bc396 | 6645 | |
2a8d363a AD |
6646 | '/' [reduce using rule 3 (exp)] |
6647 | $default reduce using rule 3 (exp) | |
ec3bc396 AD |
6648 | |
6649 | state 11 | |
6650 | ||
6651 | exp -> exp . '+' exp (rule 1) | |
6652 | exp -> exp . '-' exp (rule 2) | |
6653 | exp -> exp . '*' exp (rule 3) | |
6654 | exp -> exp . '/' exp (rule 4) | |
6655 | exp -> exp '/' exp . (rule 4) | |
6656 | ||
2a8d363a AD |
6657 | '+' shift, and go to state 4 |
6658 | '-' shift, and go to state 5 | |
6659 | '*' shift, and go to state 6 | |
6660 | '/' shift, and go to state 7 | |
ec3bc396 | 6661 | |
2a8d363a AD |
6662 | '+' [reduce using rule 4 (exp)] |
6663 | '-' [reduce using rule 4 (exp)] | |
6664 | '*' [reduce using rule 4 (exp)] | |
6665 | '/' [reduce using rule 4 (exp)] | |
6666 | $default reduce using rule 4 (exp) | |
ec3bc396 AD |
6667 | @end example |
6668 | ||
6669 | @noindent | |
fa7e68c3 PE |
6670 | Observe that state 11 contains conflicts not only due to the lack of |
6671 | precedence of @samp{/} with respect to @samp{+}, @samp{-}, and | |
6672 | @samp{*}, but also because the | |
ec3bc396 AD |
6673 | associativity of @samp{/} is not specified. |
6674 | ||
6675 | ||
6676 | @node Tracing | |
6677 | @section Tracing Your Parser | |
bfa74976 RS |
6678 | @findex yydebug |
6679 | @cindex debugging | |
6680 | @cindex tracing the parser | |
6681 | ||
6682 | If a Bison grammar compiles properly but doesn't do what you want when it | |
6683 | runs, the @code{yydebug} parser-trace feature can help you figure out why. | |
6684 | ||
3ded9a63 AD |
6685 | There are several means to enable compilation of trace facilities: |
6686 | ||
6687 | @table @asis | |
6688 | @item the macro @code{YYDEBUG} | |
6689 | @findex YYDEBUG | |
6690 | Define the macro @code{YYDEBUG} to a nonzero value when you compile the | |
c827f760 | 6691 | parser. This is compliant with @acronym{POSIX} Yacc. You could use |
3ded9a63 AD |
6692 | @samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define |
6693 | YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The | |
6694 | Prologue}). | |
6695 | ||
6696 | @item the option @option{-t}, @option{--debug} | |
6697 | Use the @samp{-t} option when you run Bison (@pxref{Invocation, | |
c827f760 | 6698 | ,Invoking Bison}). This is @acronym{POSIX} compliant too. |
3ded9a63 AD |
6699 | |
6700 | @item the directive @samp{%debug} | |
6701 | @findex %debug | |
6702 | Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison | |
6703 | Declaration Summary}). This is a Bison extension, which will prove | |
6704 | useful when Bison will output parsers for languages that don't use a | |
c827f760 PE |
6705 | preprocessor. Unless @acronym{POSIX} and Yacc portability matter to |
6706 | you, this is | |
3ded9a63 AD |
6707 | the preferred solution. |
6708 | @end table | |
6709 | ||
6710 | We suggest that you always enable the debug option so that debugging is | |
6711 | always possible. | |
bfa74976 | 6712 | |
02a81e05 | 6713 | The trace facility outputs messages with macro calls of the form |
e2742e46 | 6714 | @code{YYFPRINTF (stderr, @var{format}, @var{args})} where |
02a81e05 | 6715 | @var{format} and @var{args} are the usual @code{printf} format and |
4947ebdb PE |
6716 | arguments. If you define @code{YYDEBUG} to a nonzero value but do not |
6717 | define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included | |
e4e1a4dc | 6718 | and @code{YYPRINTF} is defined to @code{fprintf}. |
bfa74976 RS |
6719 | |
6720 | Once you have compiled the program with trace facilities, the way to | |
6721 | request a trace is to store a nonzero value in the variable @code{yydebug}. | |
6722 | You can do this by making the C code do it (in @code{main}, perhaps), or | |
6723 | you can alter the value with a C debugger. | |
6724 | ||
6725 | Each step taken by the parser when @code{yydebug} is nonzero produces a | |
6726 | line or two of trace information, written on @code{stderr}. The trace | |
6727 | messages tell you these things: | |
6728 | ||
6729 | @itemize @bullet | |
6730 | @item | |
6731 | Each time the parser calls @code{yylex}, what kind of token was read. | |
6732 | ||
6733 | @item | |
6734 | Each time a token is shifted, the depth and complete contents of the | |
6735 | state stack (@pxref{Parser States}). | |
6736 | ||
6737 | @item | |
6738 | Each time a rule is reduced, which rule it is, and the complete contents | |
6739 | of the state stack afterward. | |
6740 | @end itemize | |
6741 | ||
6742 | To make sense of this information, it helps to refer to the listing file | |
704a47c4 AD |
6743 | produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking |
6744 | Bison}). This file shows the meaning of each state in terms of | |
6745 | positions in various rules, and also what each state will do with each | |
6746 | possible input token. As you read the successive trace messages, you | |
6747 | can see that the parser is functioning according to its specification in | |
6748 | the listing file. Eventually you will arrive at the place where | |
6749 | something undesirable happens, and you will see which parts of the | |
6750 | grammar are to blame. | |
bfa74976 RS |
6751 | |
6752 | The parser file is a C program and you can use C debuggers on it, but it's | |
6753 | not easy to interpret what it is doing. The parser function is a | |
6754 | finite-state machine interpreter, and aside from the actions it executes | |
6755 | the same code over and over. Only the values of variables show where in | |
6756 | the grammar it is working. | |
6757 | ||
6758 | @findex YYPRINT | |
6759 | The debugging information normally gives the token type of each token | |
6760 | read, but not its semantic value. You can optionally define a macro | |
6761 | named @code{YYPRINT} to provide a way to print the value. If you define | |
6762 | @code{YYPRINT}, it should take three arguments. The parser will pass a | |
6763 | standard I/O stream, the numeric code for the token type, and the token | |
6764 | value (from @code{yylval}). | |
6765 | ||
6766 | Here is an example of @code{YYPRINT} suitable for the multi-function | |
6767 | calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}): | |
6768 | ||
6769 | @smallexample | |
38a92d50 PE |
6770 | %@{ |
6771 | static void print_token_value (FILE *, int, YYSTYPE); | |
6772 | #define YYPRINT(file, type, value) print_token_value (file, type, value) | |
6773 | %@} | |
6774 | ||
6775 | @dots{} %% @dots{} %% @dots{} | |
bfa74976 RS |
6776 | |
6777 | static void | |
831d3c99 | 6778 | print_token_value (FILE *file, int type, YYSTYPE value) |
bfa74976 RS |
6779 | @{ |
6780 | if (type == VAR) | |
d3c4e709 | 6781 | fprintf (file, "%s", value.tptr->name); |
bfa74976 | 6782 | else if (type == NUM) |
d3c4e709 | 6783 | fprintf (file, "%d", value.val); |
bfa74976 RS |
6784 | @} |
6785 | @end smallexample | |
6786 | ||
ec3bc396 AD |
6787 | @c ================================================= Invoking Bison |
6788 | ||
342b8b6e | 6789 | @node Invocation |
bfa74976 RS |
6790 | @chapter Invoking Bison |
6791 | @cindex invoking Bison | |
6792 | @cindex Bison invocation | |
6793 | @cindex options for invoking Bison | |
6794 | ||
6795 | The usual way to invoke Bison is as follows: | |
6796 | ||
6797 | @example | |
6798 | bison @var{infile} | |
6799 | @end example | |
6800 | ||
6801 | Here @var{infile} is the grammar file name, which usually ends in | |
6802 | @samp{.y}. The parser file's name is made by replacing the @samp{.y} | |
fa4d969f PE |
6803 | with @samp{.tab.c} and removing any leading directory. Thus, the |
6804 | @samp{bison foo.y} file name yields | |
6805 | @file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields | |
6806 | @file{foo.tab.c}. It's also possible, in case you are writing | |
79282c6c | 6807 | C++ code instead of C in your grammar file, to name it @file{foo.ypp} |
72d2299c PE |
6808 | or @file{foo.y++}. Then, the output files will take an extension like |
6809 | the given one as input (respectively @file{foo.tab.cpp} and | |
6810 | @file{foo.tab.c++}). | |
fa4d969f | 6811 | This feature takes effect with all options that manipulate file names like |
234a3be3 AD |
6812 | @samp{-o} or @samp{-d}. |
6813 | ||
6814 | For example : | |
6815 | ||
6816 | @example | |
6817 | bison -d @var{infile.yxx} | |
6818 | @end example | |
84163231 | 6819 | @noindent |
72d2299c | 6820 | will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and |
234a3be3 AD |
6821 | |
6822 | @example | |
b56471a6 | 6823 | bison -d -o @var{output.c++} @var{infile.y} |
234a3be3 | 6824 | @end example |
84163231 | 6825 | @noindent |
234a3be3 AD |
6826 | will produce @file{output.c++} and @file{outfile.h++}. |
6827 | ||
397ec073 PE |
6828 | For compatibility with @acronym{POSIX}, the standard Bison |
6829 | distribution also contains a shell script called @command{yacc} that | |
6830 | invokes Bison with the @option{-y} option. | |
6831 | ||
bfa74976 | 6832 | @menu |
13863333 | 6833 | * Bison Options:: All the options described in detail, |
c827f760 | 6834 | in alphabetical order by short options. |
bfa74976 | 6835 | * Option Cross Key:: Alphabetical list of long options. |
93dd49ab | 6836 | * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}. |
bfa74976 RS |
6837 | @end menu |
6838 | ||
342b8b6e | 6839 | @node Bison Options |
bfa74976 RS |
6840 | @section Bison Options |
6841 | ||
6842 | Bison supports both traditional single-letter options and mnemonic long | |
6843 | option names. Long option names are indicated with @samp{--} instead of | |
6844 | @samp{-}. Abbreviations for option names are allowed as long as they | |
6845 | are unique. When a long option takes an argument, like | |
6846 | @samp{--file-prefix}, connect the option name and the argument with | |
6847 | @samp{=}. | |
6848 | ||
6849 | Here is a list of options that can be used with Bison, alphabetized by | |
6850 | short option. It is followed by a cross key alphabetized by long | |
6851 | option. | |
6852 | ||
89cab50d AD |
6853 | @c Please, keep this ordered as in `bison --help'. |
6854 | @noindent | |
6855 | Operations modes: | |
6856 | @table @option | |
6857 | @item -h | |
6858 | @itemx --help | |
6859 | Print a summary of the command-line options to Bison and exit. | |
bfa74976 | 6860 | |
89cab50d AD |
6861 | @item -V |
6862 | @itemx --version | |
6863 | Print the version number of Bison and exit. | |
bfa74976 | 6864 | |
f7ab6a50 PE |
6865 | @item --print-localedir |
6866 | Print the name of the directory containing locale-dependent data. | |
6867 | ||
89cab50d AD |
6868 | @item -y |
6869 | @itemx --yacc | |
54662697 PE |
6870 | Act more like the traditional Yacc command. This can cause |
6871 | different diagnostics to be generated, and may change behavior in | |
6872 | other minor ways. Most importantly, imitate Yacc's output | |
6873 | file name conventions, so that the parser output file is called | |
89cab50d | 6874 | @file{y.tab.c}, and the other outputs are called @file{y.output} and |
b931235e JD |
6875 | @file{y.tab.h}. |
6876 | Also, if generating an @acronym{LALR}(1) parser in C, generate @code{#define} | |
6877 | statements in addition to an @code{enum} to associate token numbers with token | |
6878 | names. | |
6879 | Thus, the following shell script can substitute for Yacc, and the Bison | |
6880 | distribution contains such a script for compatibility with @acronym{POSIX}: | |
bfa74976 | 6881 | |
89cab50d | 6882 | @example |
397ec073 | 6883 | #! /bin/sh |
26e06a21 | 6884 | bison -y "$@@" |
89cab50d | 6885 | @end example |
54662697 PE |
6886 | |
6887 | The @option{-y}/@option{--yacc} option is intended for use with | |
6888 | traditional Yacc grammars. If your grammar uses a Bison extension | |
6889 | like @samp{%glr-parser}, Bison might not be Yacc-compatible even if | |
6890 | this option is specified. | |
6891 | ||
89cab50d AD |
6892 | @end table |
6893 | ||
6894 | @noindent | |
6895 | Tuning the parser: | |
6896 | ||
6897 | @table @option | |
cd5bd6ac AD |
6898 | @item -S @var{file} |
6899 | @itemx --skeleton=@var{file} | |
6900 | Specify the skeleton to use. You probably don't need this option unless | |
6901 | you are developing Bison. | |
6902 | ||
89cab50d AD |
6903 | @item -t |
6904 | @itemx --debug | |
4947ebdb PE |
6905 | In the parser file, define the macro @code{YYDEBUG} to 1 if it is not |
6906 | already defined, so that the debugging facilities are compiled. | |
ec3bc396 | 6907 | @xref{Tracing, ,Tracing Your Parser}. |
89cab50d AD |
6908 | |
6909 | @item --locations | |
d8988b2f | 6910 | Pretend that @code{%locations} was specified. @xref{Decl Summary}. |
89cab50d AD |
6911 | |
6912 | @item -p @var{prefix} | |
6913 | @itemx --name-prefix=@var{prefix} | |
d8988b2f AD |
6914 | Pretend that @code{%name-prefix="@var{prefix}"} was specified. |
6915 | @xref{Decl Summary}. | |
bfa74976 RS |
6916 | |
6917 | @item -l | |
6918 | @itemx --no-lines | |
6919 | Don't put any @code{#line} preprocessor commands in the parser file. | |
6920 | Ordinarily Bison puts them in the parser file so that the C compiler | |
6921 | and debuggers will associate errors with your source file, the | |
6922 | grammar file. This option causes them to associate errors with the | |
95e742f7 | 6923 | parser file, treating it as an independent source file in its own right. |
bfa74976 | 6924 | |
931c7513 RS |
6925 | @item -n |
6926 | @itemx --no-parser | |
d8988b2f | 6927 | Pretend that @code{%no-parser} was specified. @xref{Decl Summary}. |
931c7513 | 6928 | |
89cab50d AD |
6929 | @item -k |
6930 | @itemx --token-table | |
d8988b2f | 6931 | Pretend that @code{%token-table} was specified. @xref{Decl Summary}. |
89cab50d | 6932 | @end table |
bfa74976 | 6933 | |
89cab50d AD |
6934 | @noindent |
6935 | Adjust the output: | |
bfa74976 | 6936 | |
89cab50d AD |
6937 | @table @option |
6938 | @item -d | |
d8988b2f AD |
6939 | @itemx --defines |
6940 | Pretend that @code{%defines} was specified, i.e., write an extra output | |
6deb4447 | 6941 | file containing macro definitions for the token type names defined in |
4bfd5e4e | 6942 | the grammar, as well as a few other declarations. @xref{Decl Summary}. |
931c7513 | 6943 | |
342b8b6e | 6944 | @item --defines=@var{defines-file} |
d8988b2f | 6945 | Same as above, but save in the file @var{defines-file}. |
342b8b6e | 6946 | |
89cab50d AD |
6947 | @item -b @var{file-prefix} |
6948 | @itemx --file-prefix=@var{prefix} | |
aa08666d | 6949 | Pretend that @code{%file-prefix} was specified, i.e, specify prefix to use |
72d2299c | 6950 | for all Bison output file names. @xref{Decl Summary}. |
bfa74976 | 6951 | |
ec3bc396 AD |
6952 | @item -r @var{things} |
6953 | @itemx --report=@var{things} | |
6954 | Write an extra output file containing verbose description of the comma | |
6955 | separated list of @var{things} among: | |
6956 | ||
6957 | @table @code | |
6958 | @item state | |
6959 | Description of the grammar, conflicts (resolved and unresolved), and | |
c827f760 | 6960 | @acronym{LALR} automaton. |
ec3bc396 | 6961 | |
742e4900 | 6962 | @item lookahead |
ec3bc396 | 6963 | Implies @code{state} and augments the description of the automaton with |
742e4900 | 6964 | each rule's lookahead set. |
ec3bc396 AD |
6965 | |
6966 | @item itemset | |
6967 | Implies @code{state} and augments the description of the automaton with | |
6968 | the full set of items for each state, instead of its core only. | |
6969 | @end table | |
6970 | ||
bfa74976 RS |
6971 | @item -v |
6972 | @itemx --verbose | |
6deb4447 AD |
6973 | Pretend that @code{%verbose} was specified, i.e, write an extra output |
6974 | file containing verbose descriptions of the grammar and | |
72d2299c | 6975 | parser. @xref{Decl Summary}. |
bfa74976 | 6976 | |
fa4d969f PE |
6977 | @item -o @var{file} |
6978 | @itemx --output=@var{file} | |
6979 | Specify the @var{file} for the parser file. | |
bfa74976 | 6980 | |
fa4d969f | 6981 | The other output files' names are constructed from @var{file} as |
d8988b2f | 6982 | described under the @samp{-v} and @samp{-d} options. |
342b8b6e AD |
6983 | |
6984 | @item -g | |
c827f760 PE |
6985 | Output a @acronym{VCG} definition of the @acronym{LALR}(1) grammar |
6986 | automaton computed by Bison. If the grammar file is @file{foo.y}, the | |
6987 | @acronym{VCG} output file will | |
342b8b6e AD |
6988 | be @file{foo.vcg}. |
6989 | ||
6990 | @item --graph=@var{graph-file} | |
72d2299c PE |
6991 | The behavior of @var{--graph} is the same than @samp{-g}. The only |
6992 | difference is that it has an optional argument which is the name of | |
fa4d969f | 6993 | the output graph file. |
bfa74976 RS |
6994 | @end table |
6995 | ||
342b8b6e | 6996 | @node Option Cross Key |
bfa74976 RS |
6997 | @section Option Cross Key |
6998 | ||
aa08666d | 6999 | @c FIXME: How about putting the directives too? |
bfa74976 RS |
7000 | Here is a list of options, alphabetized by long option, to help you find |
7001 | the corresponding short option. | |
7002 | ||
aa08666d AD |
7003 | @multitable {@option{--defines=@var{defines-file}}} {@option{-b @var{file-prefix}XXX}} |
7004 | @headitem Long Option @tab Short Option | |
7005 | @item @option{--debug} @tab @option{-t} | |
7006 | @item @option{--defines=@var{defines-file}} @tab @option{-d} | |
7007 | @item @option{--file-prefix=@var{prefix}} @tab @option{-b @var{file-prefix}} | |
7008 | @item @option{--graph=@var{graph-file}} @tab @option{-d} | |
7009 | @item @option{--help} @tab @option{-h} | |
7010 | @item @option{--name-prefix=@var{prefix}} @tab @option{-p @var{name-prefix}} | |
7011 | @item @option{--no-lines} @tab @option{-l} | |
7012 | @item @option{--no-parser} @tab @option{-n} | |
7013 | @item @option{--output=@var{outfile}} @tab @option{-o @var{outfile}} | |
7014 | @item @option{--print-localedir} @tab | |
7015 | @item @option{--token-table} @tab @option{-k} | |
7016 | @item @option{--verbose} @tab @option{-v} | |
7017 | @item @option{--version} @tab @option{-V} | |
7018 | @item @option{--yacc} @tab @option{-y} | |
7019 | @end multitable | |
bfa74976 | 7020 | |
93dd49ab PE |
7021 | @node Yacc Library |
7022 | @section Yacc Library | |
7023 | ||
7024 | The Yacc library contains default implementations of the | |
7025 | @code{yyerror} and @code{main} functions. These default | |
7026 | implementations are normally not useful, but @acronym{POSIX} requires | |
7027 | them. To use the Yacc library, link your program with the | |
7028 | @option{-ly} option. Note that Bison's implementation of the Yacc | |
7029 | library is distributed under the terms of the @acronym{GNU} General | |
7030 | Public License (@pxref{Copying}). | |
7031 | ||
7032 | If you use the Yacc library's @code{yyerror} function, you should | |
7033 | declare @code{yyerror} as follows: | |
7034 | ||
7035 | @example | |
7036 | int yyerror (char const *); | |
7037 | @end example | |
7038 | ||
7039 | Bison ignores the @code{int} value returned by this @code{yyerror}. | |
7040 | If you use the Yacc library's @code{main} function, your | |
7041 | @code{yyparse} function should have the following type signature: | |
7042 | ||
7043 | @example | |
7044 | int yyparse (void); | |
7045 | @end example | |
7046 | ||
12545799 AD |
7047 | @c ================================================= C++ Bison |
7048 | ||
7049 | @node C++ Language Interface | |
7050 | @chapter C++ Language Interface | |
7051 | ||
7052 | @menu | |
7053 | * C++ Parsers:: The interface to generate C++ parser classes | |
7054 | * A Complete C++ Example:: Demonstrating their use | |
7055 | @end menu | |
7056 | ||
7057 | @node C++ Parsers | |
7058 | @section C++ Parsers | |
7059 | ||
7060 | @menu | |
7061 | * C++ Bison Interface:: Asking for C++ parser generation | |
7062 | * C++ Semantic Values:: %union vs. C++ | |
7063 | * C++ Location Values:: The position and location classes | |
7064 | * C++ Parser Interface:: Instantiating and running the parser | |
7065 | * C++ Scanner Interface:: Exchanges between yylex and parse | |
7066 | @end menu | |
7067 | ||
7068 | @node C++ Bison Interface | |
7069 | @subsection C++ Bison Interface | |
7070 | @c - %skeleton "lalr1.cc" | |
7071 | @c - Always pure | |
7072 | @c - initial action | |
7073 | ||
aa08666d AD |
7074 | The C++ parser @acronym{LALR}(1) skeleton is named @file{lalr1.cc}. To |
7075 | select it, you may either pass the option @option{--skeleton=lalr1.cc} | |
7076 | to Bison, or include the directive @samp{%skeleton "lalr1.cc"} in the | |
12545799 | 7077 | grammar preamble. When run, @command{bison} will create several |
aa08666d AD |
7078 | entities in the @samp{yy} namespace. Use the @samp{%name-prefix} |
7079 | directive to change the namespace name, see @ref{Decl Summary}. The | |
7080 | various classes are generated in the following files: | |
7081 | ||
12545799 AD |
7082 | @table @file |
7083 | @item position.hh | |
7084 | @itemx location.hh | |
7085 | The definition of the classes @code{position} and @code{location}, | |
7086 | used for location tracking. @xref{C++ Location Values}. | |
7087 | ||
7088 | @item stack.hh | |
7089 | An auxiliary class @code{stack} used by the parser. | |
7090 | ||
fa4d969f PE |
7091 | @item @var{file}.hh |
7092 | @itemx @var{file}.cc | |
cd8b5791 AD |
7093 | (Assuming the extension of the input file was @samp{.yy}.) The |
7094 | declaration and implementation of the C++ parser class. The basename | |
7095 | and extension of these two files follow the same rules as with regular C | |
7096 | parsers (@pxref{Invocation}). | |
12545799 | 7097 | |
cd8b5791 AD |
7098 | The header is @emph{mandatory}; you must either pass |
7099 | @option{-d}/@option{--defines} to @command{bison}, or use the | |
12545799 AD |
7100 | @samp{%defines} directive. |
7101 | @end table | |
7102 | ||
7103 | All these files are documented using Doxygen; run @command{doxygen} | |
7104 | for a complete and accurate documentation. | |
7105 | ||
7106 | @node C++ Semantic Values | |
7107 | @subsection C++ Semantic Values | |
7108 | @c - No objects in unions | |
7109 | @c - YSTYPE | |
7110 | @c - Printer and destructor | |
7111 | ||
7112 | The @code{%union} directive works as for C, see @ref{Union Decl, ,The | |
7113 | Collection of Value Types}. In particular it produces a genuine | |
7114 | @code{union}@footnote{In the future techniques to allow complex types | |
fb9712a9 AD |
7115 | within pseudo-unions (similar to Boost variants) might be implemented to |
7116 | alleviate these issues.}, which have a few specific features in C++. | |
12545799 AD |
7117 | @itemize @minus |
7118 | @item | |
fb9712a9 AD |
7119 | The type @code{YYSTYPE} is defined but its use is discouraged: rather |
7120 | you should refer to the parser's encapsulated type | |
7121 | @code{yy::parser::semantic_type}. | |
12545799 AD |
7122 | @item |
7123 | Non POD (Plain Old Data) types cannot be used. C++ forbids any | |
7124 | instance of classes with constructors in unions: only @emph{pointers} | |
7125 | to such objects are allowed. | |
7126 | @end itemize | |
7127 | ||
7128 | Because objects have to be stored via pointers, memory is not | |
7129 | reclaimed automatically: using the @code{%destructor} directive is the | |
7130 | only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded | |
7131 | Symbols}. | |
7132 | ||
7133 | ||
7134 | @node C++ Location Values | |
7135 | @subsection C++ Location Values | |
7136 | @c - %locations | |
7137 | @c - class Position | |
7138 | @c - class Location | |
b47dbebe | 7139 | @c - %define "filename_type" "const symbol::Symbol" |
12545799 AD |
7140 | |
7141 | When the directive @code{%locations} is used, the C++ parser supports | |
7142 | location tracking, see @ref{Locations, , Locations Overview}. Two | |
7143 | auxiliary classes define a @code{position}, a single point in a file, | |
7144 | and a @code{location}, a range composed of a pair of | |
7145 | @code{position}s (possibly spanning several files). | |
7146 | ||
fa4d969f | 7147 | @deftypemethod {position} {std::string*} file |
12545799 AD |
7148 | The name of the file. It will always be handled as a pointer, the |
7149 | parser will never duplicate nor deallocate it. As an experimental | |
7150 | feature you may change it to @samp{@var{type}*} using @samp{%define | |
b47dbebe | 7151 | "filename_type" "@var{type}"}. |
12545799 AD |
7152 | @end deftypemethod |
7153 | ||
7154 | @deftypemethod {position} {unsigned int} line | |
7155 | The line, starting at 1. | |
7156 | @end deftypemethod | |
7157 | ||
7158 | @deftypemethod {position} {unsigned int} lines (int @var{height} = 1) | |
7159 | Advance by @var{height} lines, resetting the column number. | |
7160 | @end deftypemethod | |
7161 | ||
7162 | @deftypemethod {position} {unsigned int} column | |
7163 | The column, starting at 0. | |
7164 | @end deftypemethod | |
7165 | ||
7166 | @deftypemethod {position} {unsigned int} columns (int @var{width} = 1) | |
7167 | Advance by @var{width} columns, without changing the line number. | |
7168 | @end deftypemethod | |
7169 | ||
7170 | @deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width}) | |
7171 | @deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width}) | |
7172 | @deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width}) | |
7173 | @deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width}) | |
7174 | Various forms of syntactic sugar for @code{columns}. | |
7175 | @end deftypemethod | |
7176 | ||
7177 | @deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p}) | |
7178 | Report @var{p} on @var{o} like this: | |
fa4d969f PE |
7179 | @samp{@var{file}:@var{line}.@var{column}}, or |
7180 | @samp{@var{line}.@var{column}} if @var{file} is null. | |
12545799 AD |
7181 | @end deftypemethod |
7182 | ||
7183 | @deftypemethod {location} {position} begin | |
7184 | @deftypemethodx {location} {position} end | |
7185 | The first, inclusive, position of the range, and the first beyond. | |
7186 | @end deftypemethod | |
7187 | ||
7188 | @deftypemethod {location} {unsigned int} columns (int @var{width} = 1) | |
7189 | @deftypemethodx {location} {unsigned int} lines (int @var{height} = 1) | |
7190 | Advance the @code{end} position. | |
7191 | @end deftypemethod | |
7192 | ||
7193 | @deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end}) | |
7194 | @deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width}) | |
7195 | @deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width}) | |
7196 | Various forms of syntactic sugar. | |
7197 | @end deftypemethod | |
7198 | ||
7199 | @deftypemethod {location} {void} step () | |
7200 | Move @code{begin} onto @code{end}. | |
7201 | @end deftypemethod | |
7202 | ||
7203 | ||
7204 | @node C++ Parser Interface | |
7205 | @subsection C++ Parser Interface | |
7206 | @c - define parser_class_name | |
7207 | @c - Ctor | |
7208 | @c - parse, error, set_debug_level, debug_level, set_debug_stream, | |
7209 | @c debug_stream. | |
7210 | @c - Reporting errors | |
7211 | ||
7212 | The output files @file{@var{output}.hh} and @file{@var{output}.cc} | |
7213 | declare and define the parser class in the namespace @code{yy}. The | |
7214 | class name defaults to @code{parser}, but may be changed using | |
7215 | @samp{%define "parser_class_name" "@var{name}"}. The interface of | |
9d9b8b70 | 7216 | this class is detailed below. It can be extended using the |
12545799 AD |
7217 | @code{%parse-param} feature: its semantics is slightly changed since |
7218 | it describes an additional member of the parser class, and an | |
7219 | additional argument for its constructor. | |
7220 | ||
8a0adb01 AD |
7221 | @defcv {Type} {parser} {semantic_value_type} |
7222 | @defcvx {Type} {parser} {location_value_type} | |
12545799 | 7223 | The types for semantics value and locations. |
8a0adb01 | 7224 | @end defcv |
12545799 AD |
7225 | |
7226 | @deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...) | |
7227 | Build a new parser object. There are no arguments by default, unless | |
7228 | @samp{%parse-param @{@var{type1} @var{arg1}@}} was used. | |
7229 | @end deftypemethod | |
7230 | ||
7231 | @deftypemethod {parser} {int} parse () | |
7232 | Run the syntactic analysis, and return 0 on success, 1 otherwise. | |
7233 | @end deftypemethod | |
7234 | ||
7235 | @deftypemethod {parser} {std::ostream&} debug_stream () | |
7236 | @deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o}) | |
7237 | Get or set the stream used for tracing the parsing. It defaults to | |
7238 | @code{std::cerr}. | |
7239 | @end deftypemethod | |
7240 | ||
7241 | @deftypemethod {parser} {debug_level_type} debug_level () | |
7242 | @deftypemethodx {parser} {void} set_debug_level (debug_level @var{l}) | |
7243 | Get or set the tracing level. Currently its value is either 0, no trace, | |
9d9b8b70 | 7244 | or nonzero, full tracing. |
12545799 AD |
7245 | @end deftypemethod |
7246 | ||
7247 | @deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m}) | |
7248 | The definition for this member function must be supplied by the user: | |
7249 | the parser uses it to report a parser error occurring at @var{l}, | |
7250 | described by @var{m}. | |
7251 | @end deftypemethod | |
7252 | ||
7253 | ||
7254 | @node C++ Scanner Interface | |
7255 | @subsection C++ Scanner Interface | |
7256 | @c - prefix for yylex. | |
7257 | @c - Pure interface to yylex | |
7258 | @c - %lex-param | |
7259 | ||
7260 | The parser invokes the scanner by calling @code{yylex}. Contrary to C | |
7261 | parsers, C++ parsers are always pure: there is no point in using the | |
7262 | @code{%pure-parser} directive. Therefore the interface is as follows. | |
7263 | ||
7264 | @deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...) | |
7265 | Return the next token. Its type is the return value, its semantic | |
7266 | value and location being @var{yylval} and @var{yylloc}. Invocations of | |
7267 | @samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments. | |
7268 | @end deftypemethod | |
7269 | ||
7270 | ||
7271 | @node A Complete C++ Example | |
7272 | @section A Complete C++ Example | |
7273 | ||
7274 | This section demonstrates the use of a C++ parser with a simple but | |
7275 | complete example. This example should be available on your system, | |
7276 | ready to compile, in the directory @dfn{../bison/examples/calc++}. It | |
7277 | focuses on the use of Bison, therefore the design of the various C++ | |
7278 | classes is very naive: no accessors, no encapsulation of members etc. | |
7279 | We will use a Lex scanner, and more precisely, a Flex scanner, to | |
7280 | demonstrate the various interaction. A hand written scanner is | |
7281 | actually easier to interface with. | |
7282 | ||
7283 | @menu | |
7284 | * Calc++ --- C++ Calculator:: The specifications | |
7285 | * Calc++ Parsing Driver:: An active parsing context | |
7286 | * Calc++ Parser:: A parser class | |
7287 | * Calc++ Scanner:: A pure C++ Flex scanner | |
7288 | * Calc++ Top Level:: Conducting the band | |
7289 | @end menu | |
7290 | ||
7291 | @node Calc++ --- C++ Calculator | |
7292 | @subsection Calc++ --- C++ Calculator | |
7293 | ||
7294 | Of course the grammar is dedicated to arithmetics, a single | |
9d9b8b70 | 7295 | expression, possibly preceded by variable assignments. An |
12545799 AD |
7296 | environment containing possibly predefined variables such as |
7297 | @code{one} and @code{two}, is exchanged with the parser. An example | |
7298 | of valid input follows. | |
7299 | ||
7300 | @example | |
7301 | three := 3 | |
7302 | seven := one + two * three | |
7303 | seven * seven | |
7304 | @end example | |
7305 | ||
7306 | @node Calc++ Parsing Driver | |
7307 | @subsection Calc++ Parsing Driver | |
7308 | @c - An env | |
7309 | @c - A place to store error messages | |
7310 | @c - A place for the result | |
7311 | ||
7312 | To support a pure interface with the parser (and the scanner) the | |
7313 | technique of the ``parsing context'' is convenient: a structure | |
7314 | containing all the data to exchange. Since, in addition to simply | |
7315 | launch the parsing, there are several auxiliary tasks to execute (open | |
7316 | the file for parsing, instantiate the parser etc.), we recommend | |
7317 | transforming the simple parsing context structure into a fully blown | |
7318 | @dfn{parsing driver} class. | |
7319 | ||
7320 | The declaration of this driver class, @file{calc++-driver.hh}, is as | |
7321 | follows. The first part includes the CPP guard and imports the | |
fb9712a9 AD |
7322 | required standard library components, and the declaration of the parser |
7323 | class. | |
12545799 | 7324 | |
1c59e0a1 | 7325 | @comment file: calc++-driver.hh |
12545799 AD |
7326 | @example |
7327 | #ifndef CALCXX_DRIVER_HH | |
7328 | # define CALCXX_DRIVER_HH | |
7329 | # include <string> | |
7330 | # include <map> | |
fb9712a9 | 7331 | # include "calc++-parser.hh" |
12545799 AD |
7332 | @end example |
7333 | ||
12545799 AD |
7334 | |
7335 | @noindent | |
7336 | Then comes the declaration of the scanning function. Flex expects | |
7337 | the signature of @code{yylex} to be defined in the macro | |
7338 | @code{YY_DECL}, and the C++ parser expects it to be declared. We can | |
7339 | factor both as follows. | |
1c59e0a1 AD |
7340 | |
7341 | @comment file: calc++-driver.hh | |
12545799 AD |
7342 | @example |
7343 | // Announce to Flex the prototype we want for lexing function, ... | |
c095d689 AD |
7344 | # define YY_DECL \ |
7345 | yy::calcxx_parser::token_type \ | |
7346 | yylex (yy::calcxx_parser::semantic_type* yylval, \ | |
7347 | yy::calcxx_parser::location_type* yylloc, \ | |
7348 | calcxx_driver& driver) | |
12545799 AD |
7349 | // ... and declare it for the parser's sake. |
7350 | YY_DECL; | |
7351 | @end example | |
7352 | ||
7353 | @noindent | |
7354 | The @code{calcxx_driver} class is then declared with its most obvious | |
7355 | members. | |
7356 | ||
1c59e0a1 | 7357 | @comment file: calc++-driver.hh |
12545799 AD |
7358 | @example |
7359 | // Conducting the whole scanning and parsing of Calc++. | |
7360 | class calcxx_driver | |
7361 | @{ | |
7362 | public: | |
7363 | calcxx_driver (); | |
7364 | virtual ~calcxx_driver (); | |
7365 | ||
7366 | std::map<std::string, int> variables; | |
7367 | ||
7368 | int result; | |
7369 | @end example | |
7370 | ||
7371 | @noindent | |
7372 | To encapsulate the coordination with the Flex scanner, it is useful to | |
7373 | have two members function to open and close the scanning phase. | |
7374 | members. | |
7375 | ||
1c59e0a1 | 7376 | @comment file: calc++-driver.hh |
12545799 AD |
7377 | @example |
7378 | // Handling the scanner. | |
7379 | void scan_begin (); | |
7380 | void scan_end (); | |
7381 | bool trace_scanning; | |
7382 | @end example | |
7383 | ||
7384 | @noindent | |
7385 | Similarly for the parser itself. | |
7386 | ||
1c59e0a1 | 7387 | @comment file: calc++-driver.hh |
12545799 AD |
7388 | @example |
7389 | // Handling the parser. | |
7390 | void parse (const std::string& f); | |
7391 | std::string file; | |
7392 | bool trace_parsing; | |
7393 | @end example | |
7394 | ||
7395 | @noindent | |
7396 | To demonstrate pure handling of parse errors, instead of simply | |
7397 | dumping them on the standard error output, we will pass them to the | |
7398 | compiler driver using the following two member functions. Finally, we | |
7399 | close the class declaration and CPP guard. | |
7400 | ||
1c59e0a1 | 7401 | @comment file: calc++-driver.hh |
12545799 AD |
7402 | @example |
7403 | // Error handling. | |
7404 | void error (const yy::location& l, const std::string& m); | |
7405 | void error (const std::string& m); | |
7406 | @}; | |
7407 | #endif // ! CALCXX_DRIVER_HH | |
7408 | @end example | |
7409 | ||
7410 | The implementation of the driver is straightforward. The @code{parse} | |
7411 | member function deserves some attention. The @code{error} functions | |
7412 | are simple stubs, they should actually register the located error | |
7413 | messages and set error state. | |
7414 | ||
1c59e0a1 | 7415 | @comment file: calc++-driver.cc |
12545799 AD |
7416 | @example |
7417 | #include "calc++-driver.hh" | |
7418 | #include "calc++-parser.hh" | |
7419 | ||
7420 | calcxx_driver::calcxx_driver () | |
7421 | : trace_scanning (false), trace_parsing (false) | |
7422 | @{ | |
7423 | variables["one"] = 1; | |
7424 | variables["two"] = 2; | |
7425 | @} | |
7426 | ||
7427 | calcxx_driver::~calcxx_driver () | |
7428 | @{ | |
7429 | @} | |
7430 | ||
7431 | void | |
7432 | calcxx_driver::parse (const std::string &f) | |
7433 | @{ | |
7434 | file = f; | |
7435 | scan_begin (); | |
7436 | yy::calcxx_parser parser (*this); | |
7437 | parser.set_debug_level (trace_parsing); | |
7438 | parser.parse (); | |
7439 | scan_end (); | |
7440 | @} | |
7441 | ||
7442 | void | |
7443 | calcxx_driver::error (const yy::location& l, const std::string& m) | |
7444 | @{ | |
7445 | std::cerr << l << ": " << m << std::endl; | |
7446 | @} | |
7447 | ||
7448 | void | |
7449 | calcxx_driver::error (const std::string& m) | |
7450 | @{ | |
7451 | std::cerr << m << std::endl; | |
7452 | @} | |
7453 | @end example | |
7454 | ||
7455 | @node Calc++ Parser | |
7456 | @subsection Calc++ Parser | |
7457 | ||
b50d2359 AD |
7458 | The parser definition file @file{calc++-parser.yy} starts by asking for |
7459 | the C++ LALR(1) skeleton, the creation of the parser header file, and | |
7460 | specifies the name of the parser class. Because the C++ skeleton | |
7461 | changed several times, it is safer to require the version you designed | |
7462 | the grammar for. | |
1c59e0a1 AD |
7463 | |
7464 | @comment file: calc++-parser.yy | |
12545799 AD |
7465 | @example |
7466 | %skeleton "lalr1.cc" /* -*- C++ -*- */ | |
b50d2359 | 7467 | %require "2.1a" |
12545799 | 7468 | %defines |
fb9712a9 AD |
7469 | %define "parser_class_name" "calcxx_parser" |
7470 | @end example | |
7471 | ||
7472 | @noindent | |
34f98f46 | 7473 | @findex %start-header |
fb9712a9 AD |
7474 | Then come the declarations/inclusions needed to define the |
7475 | @code{%union}. Because the parser uses the parsing driver and | |
7476 | reciprocally, both cannot include the header of the other. Because the | |
7477 | driver's header needs detailed knowledge about the parser class (in | |
7478 | particular its inner types), it is the parser's header which will simply | |
7479 | use a forward declaration of the driver. | |
34f98f46 | 7480 | @xref{Table of Symbols, ,%start-header}. |
fb9712a9 AD |
7481 | |
7482 | @comment file: calc++-parser.yy | |
7483 | @example | |
34f98f46 | 7484 | %start-header @{ |
12545799 | 7485 | # include <string> |
fb9712a9 | 7486 | class calcxx_driver; |
9bc0dd67 | 7487 | @} |
12545799 AD |
7488 | @end example |
7489 | ||
7490 | @noindent | |
7491 | The driver is passed by reference to the parser and to the scanner. | |
7492 | This provides a simple but effective pure interface, not relying on | |
7493 | global variables. | |
7494 | ||
1c59e0a1 | 7495 | @comment file: calc++-parser.yy |
12545799 AD |
7496 | @example |
7497 | // The parsing context. | |
7498 | %parse-param @{ calcxx_driver& driver @} | |
7499 | %lex-param @{ calcxx_driver& driver @} | |
7500 | @end example | |
7501 | ||
7502 | @noindent | |
7503 | Then we request the location tracking feature, and initialize the | |
7504 | first location's file name. Afterwards new locations are computed | |
7505 | relatively to the previous locations: the file name will be | |
7506 | automatically propagated. | |
7507 | ||
1c59e0a1 | 7508 | @comment file: calc++-parser.yy |
12545799 AD |
7509 | @example |
7510 | %locations | |
7511 | %initial-action | |
7512 | @{ | |
7513 | // Initialize the initial location. | |
b47dbebe | 7514 | @@$.begin.filename = @@$.end.filename = &driver.file; |
12545799 AD |
7515 | @}; |
7516 | @end example | |
7517 | ||
7518 | @noindent | |
7519 | Use the two following directives to enable parser tracing and verbose | |
7520 | error messages. | |
7521 | ||
1c59e0a1 | 7522 | @comment file: calc++-parser.yy |
12545799 AD |
7523 | @example |
7524 | %debug | |
7525 | %error-verbose | |
7526 | @end example | |
7527 | ||
7528 | @noindent | |
7529 | Semantic values cannot use ``real'' objects, but only pointers to | |
7530 | them. | |
7531 | ||
1c59e0a1 | 7532 | @comment file: calc++-parser.yy |
12545799 AD |
7533 | @example |
7534 | // Symbols. | |
7535 | %union | |
7536 | @{ | |
7537 | int ival; | |
7538 | std::string *sval; | |
7539 | @}; | |
7540 | @end example | |
7541 | ||
fb9712a9 | 7542 | @noindent |
34f98f46 JD |
7543 | @findex %after-header |
7544 | The code between @samp{%after-header @{} and @samp{@}} is output in the | |
7545 | @file{*.cc} file; it needs detailed knowledge about the driver. | |
fb9712a9 AD |
7546 | |
7547 | @comment file: calc++-parser.yy | |
7548 | @example | |
34f98f46 | 7549 | %after-header @{ |
fb9712a9 | 7550 | # include "calc++-driver.hh" |
34f98f46 | 7551 | @} |
fb9712a9 AD |
7552 | @end example |
7553 | ||
7554 | ||
12545799 AD |
7555 | @noindent |
7556 | The token numbered as 0 corresponds to end of file; the following line | |
7557 | allows for nicer error messages referring to ``end of file'' instead | |
7558 | of ``$end''. Similarly user friendly named are provided for each | |
7559 | symbol. Note that the tokens names are prefixed by @code{TOKEN_} to | |
7560 | avoid name clashes. | |
7561 | ||
1c59e0a1 | 7562 | @comment file: calc++-parser.yy |
12545799 | 7563 | @example |
fb9712a9 AD |
7564 | %token END 0 "end of file" |
7565 | %token ASSIGN ":=" | |
7566 | %token <sval> IDENTIFIER "identifier" | |
7567 | %token <ival> NUMBER "number" | |
7568 | %type <ival> exp "expression" | |
12545799 AD |
7569 | @end example |
7570 | ||
7571 | @noindent | |
7572 | To enable memory deallocation during error recovery, use | |
7573 | @code{%destructor}. | |
7574 | ||
287c78f6 | 7575 | @c FIXME: Document %printer, and mention that it takes a braced-code operand. |
1c59e0a1 | 7576 | @comment file: calc++-parser.yy |
12545799 AD |
7577 | @example |
7578 | %printer @{ debug_stream () << *$$; @} "identifier" | |
7579 | %destructor @{ delete $$; @} "identifier" | |
7580 | ||
7581 | %printer @{ debug_stream () << $$; @} "number" "expression" | |
7582 | @end example | |
7583 | ||
7584 | @noindent | |
7585 | The grammar itself is straightforward. | |
7586 | ||
1c59e0a1 | 7587 | @comment file: calc++-parser.yy |
12545799 AD |
7588 | @example |
7589 | %% | |
7590 | %start unit; | |
7591 | unit: assignments exp @{ driver.result = $2; @}; | |
7592 | ||
7593 | assignments: assignments assignment @{@} | |
9d9b8b70 | 7594 | | /* Nothing. */ @{@}; |
12545799 | 7595 | |
fb9712a9 | 7596 | assignment: "identifier" ":=" exp @{ driver.variables[*$1] = $3; @}; |
12545799 AD |
7597 | |
7598 | %left '+' '-'; | |
7599 | %left '*' '/'; | |
7600 | exp: exp '+' exp @{ $$ = $1 + $3; @} | |
7601 | | exp '-' exp @{ $$ = $1 - $3; @} | |
7602 | | exp '*' exp @{ $$ = $1 * $3; @} | |
7603 | | exp '/' exp @{ $$ = $1 / $3; @} | |
fb9712a9 AD |
7604 | | "identifier" @{ $$ = driver.variables[*$1]; @} |
7605 | | "number" @{ $$ = $1; @}; | |
12545799 AD |
7606 | %% |
7607 | @end example | |
7608 | ||
7609 | @noindent | |
7610 | Finally the @code{error} member function registers the errors to the | |
7611 | driver. | |
7612 | ||
1c59e0a1 | 7613 | @comment file: calc++-parser.yy |
12545799 AD |
7614 | @example |
7615 | void | |
1c59e0a1 AD |
7616 | yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l, |
7617 | const std::string& m) | |
12545799 AD |
7618 | @{ |
7619 | driver.error (l, m); | |
7620 | @} | |
7621 | @end example | |
7622 | ||
7623 | @node Calc++ Scanner | |
7624 | @subsection Calc++ Scanner | |
7625 | ||
7626 | The Flex scanner first includes the driver declaration, then the | |
7627 | parser's to get the set of defined tokens. | |
7628 | ||
1c59e0a1 | 7629 | @comment file: calc++-scanner.ll |
12545799 AD |
7630 | @example |
7631 | %@{ /* -*- C++ -*- */ | |
04098407 PE |
7632 | # include <cstdlib> |
7633 | # include <errno.h> | |
7634 | # include <limits.h> | |
12545799 AD |
7635 | # include <string> |
7636 | # include "calc++-driver.hh" | |
7637 | # include "calc++-parser.hh" | |
eaea13f5 PE |
7638 | |
7639 | /* Work around an incompatibility in flex (at least versions | |
7640 | 2.5.31 through 2.5.33): it generates code that does | |
7641 | not conform to C89. See Debian bug 333231 | |
7642 | <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */ | |
7870f699 PE |
7643 | # undef yywrap |
7644 | # define yywrap() 1 | |
eaea13f5 | 7645 | |
c095d689 AD |
7646 | /* By default yylex returns int, we use token_type. |
7647 | Unfortunately yyterminate by default returns 0, which is | |
7648 | not of token_type. */ | |
8c5b881d | 7649 | #define yyterminate() return token::END |
12545799 AD |
7650 | %@} |
7651 | @end example | |
7652 | ||
7653 | @noindent | |
7654 | Because there is no @code{#include}-like feature we don't need | |
7655 | @code{yywrap}, we don't need @code{unput} either, and we parse an | |
7656 | actual file, this is not an interactive session with the user. | |
7657 | Finally we enable the scanner tracing features. | |
7658 | ||
1c59e0a1 | 7659 | @comment file: calc++-scanner.ll |
12545799 AD |
7660 | @example |
7661 | %option noyywrap nounput batch debug | |
7662 | @end example | |
7663 | ||
7664 | @noindent | |
7665 | Abbreviations allow for more readable rules. | |
7666 | ||
1c59e0a1 | 7667 | @comment file: calc++-scanner.ll |
12545799 AD |
7668 | @example |
7669 | id [a-zA-Z][a-zA-Z_0-9]* | |
7670 | int [0-9]+ | |
7671 | blank [ \t] | |
7672 | @end example | |
7673 | ||
7674 | @noindent | |
9d9b8b70 | 7675 | The following paragraph suffices to track locations accurately. Each |
12545799 AD |
7676 | time @code{yylex} is invoked, the begin position is moved onto the end |
7677 | position. Then when a pattern is matched, the end position is | |
7678 | advanced of its width. In case it matched ends of lines, the end | |
7679 | cursor is adjusted, and each time blanks are matched, the begin cursor | |
7680 | is moved onto the end cursor to effectively ignore the blanks | |
7681 | preceding tokens. Comments would be treated equally. | |
7682 | ||
1c59e0a1 | 7683 | @comment file: calc++-scanner.ll |
12545799 | 7684 | @example |
828c373b AD |
7685 | %@{ |
7686 | # define YY_USER_ACTION yylloc->columns (yyleng); | |
7687 | %@} | |
12545799 AD |
7688 | %% |
7689 | %@{ | |
7690 | yylloc->step (); | |
12545799 AD |
7691 | %@} |
7692 | @{blank@}+ yylloc->step (); | |
7693 | [\n]+ yylloc->lines (yyleng); yylloc->step (); | |
7694 | @end example | |
7695 | ||
7696 | @noindent | |
fb9712a9 AD |
7697 | The rules are simple, just note the use of the driver to report errors. |
7698 | It is convenient to use a typedef to shorten | |
7699 | @code{yy::calcxx_parser::token::identifier} into | |
9d9b8b70 | 7700 | @code{token::identifier} for instance. |
12545799 | 7701 | |
1c59e0a1 | 7702 | @comment file: calc++-scanner.ll |
12545799 | 7703 | @example |
fb9712a9 AD |
7704 | %@{ |
7705 | typedef yy::calcxx_parser::token token; | |
7706 | %@} | |
8c5b881d | 7707 | /* Convert ints to the actual type of tokens. */ |
c095d689 | 7708 | [-+*/] return yy::calcxx_parser::token_type (yytext[0]); |
fb9712a9 | 7709 | ":=" return token::ASSIGN; |
04098407 PE |
7710 | @{int@} @{ |
7711 | errno = 0; | |
7712 | long n = strtol (yytext, NULL, 10); | |
7713 | if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE)) | |
7714 | driver.error (*yylloc, "integer is out of range"); | |
7715 | yylval->ival = n; | |
fb9712a9 | 7716 | return token::NUMBER; |
04098407 | 7717 | @} |
fb9712a9 | 7718 | @{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER; |
12545799 AD |
7719 | . driver.error (*yylloc, "invalid character"); |
7720 | %% | |
7721 | @end example | |
7722 | ||
7723 | @noindent | |
7724 | Finally, because the scanner related driver's member function depend | |
7725 | on the scanner's data, it is simpler to implement them in this file. | |
7726 | ||
1c59e0a1 | 7727 | @comment file: calc++-scanner.ll |
12545799 AD |
7728 | @example |
7729 | void | |
7730 | calcxx_driver::scan_begin () | |
7731 | @{ | |
7732 | yy_flex_debug = trace_scanning; | |
7733 | if (!(yyin = fopen (file.c_str (), "r"))) | |
7734 | error (std::string ("cannot open ") + file); | |
7735 | @} | |
7736 | ||
7737 | void | |
7738 | calcxx_driver::scan_end () | |
7739 | @{ | |
7740 | fclose (yyin); | |
7741 | @} | |
7742 | @end example | |
7743 | ||
7744 | @node Calc++ Top Level | |
7745 | @subsection Calc++ Top Level | |
7746 | ||
7747 | The top level file, @file{calc++.cc}, poses no problem. | |
7748 | ||
1c59e0a1 | 7749 | @comment file: calc++.cc |
12545799 AD |
7750 | @example |
7751 | #include <iostream> | |
7752 | #include "calc++-driver.hh" | |
7753 | ||
7754 | int | |
fa4d969f | 7755 | main (int argc, char *argv[]) |
12545799 AD |
7756 | @{ |
7757 | calcxx_driver driver; | |
7758 | for (++argv; argv[0]; ++argv) | |
7759 | if (*argv == std::string ("-p")) | |
7760 | driver.trace_parsing = true; | |
7761 | else if (*argv == std::string ("-s")) | |
7762 | driver.trace_scanning = true; | |
7763 | else | |
7764 | @{ | |
7765 | driver.parse (*argv); | |
7766 | std::cout << driver.result << std::endl; | |
7767 | @} | |
7768 | @} | |
7769 | @end example | |
7770 | ||
7771 | @c ================================================= FAQ | |
d1a1114f AD |
7772 | |
7773 | @node FAQ | |
7774 | @chapter Frequently Asked Questions | |
7775 | @cindex frequently asked questions | |
7776 | @cindex questions | |
7777 | ||
7778 | Several questions about Bison come up occasionally. Here some of them | |
7779 | are addressed. | |
7780 | ||
7781 | @menu | |
55ba27be AD |
7782 | * Memory Exhausted:: Breaking the Stack Limits |
7783 | * How Can I Reset the Parser:: @code{yyparse} Keeps some State | |
7784 | * Strings are Destroyed:: @code{yylval} Loses Track of Strings | |
7785 | * Implementing Gotos/Loops:: Control Flow in the Calculator | |
ed2e6384 | 7786 | * Multiple start-symbols:: Factoring closely related grammars |
55ba27be AD |
7787 | * Secure? Conform?:: Is Bison @acronym{POSIX} safe? |
7788 | * I can't build Bison:: Troubleshooting | |
7789 | * Where can I find help?:: Troubleshouting | |
7790 | * Bug Reports:: Troublereporting | |
7791 | * Other Languages:: Parsers in Java and others | |
7792 | * Beta Testing:: Experimenting development versions | |
7793 | * Mailing Lists:: Meeting other Bison users | |
d1a1114f AD |
7794 | @end menu |
7795 | ||
1a059451 PE |
7796 | @node Memory Exhausted |
7797 | @section Memory Exhausted | |
d1a1114f AD |
7798 | |
7799 | @display | |
1a059451 | 7800 | My parser returns with error with a @samp{memory exhausted} |
d1a1114f AD |
7801 | message. What can I do? |
7802 | @end display | |
7803 | ||
7804 | This question is already addressed elsewhere, @xref{Recursion, | |
7805 | ,Recursive Rules}. | |
7806 | ||
e64fec0a PE |
7807 | @node How Can I Reset the Parser |
7808 | @section How Can I Reset the Parser | |
5b066063 | 7809 | |
0e14ad77 PE |
7810 | The following phenomenon has several symptoms, resulting in the |
7811 | following typical questions: | |
5b066063 AD |
7812 | |
7813 | @display | |
7814 | I invoke @code{yyparse} several times, and on correct input it works | |
7815 | properly; but when a parse error is found, all the other calls fail | |
0e14ad77 | 7816 | too. How can I reset the error flag of @code{yyparse}? |
5b066063 AD |
7817 | @end display |
7818 | ||
7819 | @noindent | |
7820 | or | |
7821 | ||
7822 | @display | |
0e14ad77 | 7823 | My parser includes support for an @samp{#include}-like feature, in |
5b066063 AD |
7824 | which case I run @code{yyparse} from @code{yyparse}. This fails |
7825 | although I did specify I needed a @code{%pure-parser}. | |
7826 | @end display | |
7827 | ||
0e14ad77 PE |
7828 | These problems typically come not from Bison itself, but from |
7829 | Lex-generated scanners. Because these scanners use large buffers for | |
5b066063 AD |
7830 | speed, they might not notice a change of input file. As a |
7831 | demonstration, consider the following source file, | |
7832 | @file{first-line.l}: | |
7833 | ||
7834 | @verbatim | |
7835 | %{ | |
7836 | #include <stdio.h> | |
7837 | #include <stdlib.h> | |
7838 | %} | |
7839 | %% | |
7840 | .*\n ECHO; return 1; | |
7841 | %% | |
7842 | int | |
0e14ad77 | 7843 | yyparse (char const *file) |
5b066063 AD |
7844 | { |
7845 | yyin = fopen (file, "r"); | |
7846 | if (!yyin) | |
7847 | exit (2); | |
fa7e68c3 | 7848 | /* One token only. */ |
5b066063 | 7849 | yylex (); |
0e14ad77 | 7850 | if (fclose (yyin) != 0) |
5b066063 AD |
7851 | exit (3); |
7852 | return 0; | |
7853 | } | |
7854 | ||
7855 | int | |
0e14ad77 | 7856 | main (void) |
5b066063 AD |
7857 | { |
7858 | yyparse ("input"); | |
7859 | yyparse ("input"); | |
7860 | return 0; | |
7861 | } | |
7862 | @end verbatim | |
7863 | ||
7864 | @noindent | |
7865 | If the file @file{input} contains | |
7866 | ||
7867 | @verbatim | |
7868 | input:1: Hello, | |
7869 | input:2: World! | |
7870 | @end verbatim | |
7871 | ||
7872 | @noindent | |
0e14ad77 | 7873 | then instead of getting the first line twice, you get: |
5b066063 AD |
7874 | |
7875 | @example | |
7876 | $ @kbd{flex -ofirst-line.c first-line.l} | |
7877 | $ @kbd{gcc -ofirst-line first-line.c -ll} | |
7878 | $ @kbd{./first-line} | |
7879 | input:1: Hello, | |
7880 | input:2: World! | |
7881 | @end example | |
7882 | ||
0e14ad77 PE |
7883 | Therefore, whenever you change @code{yyin}, you must tell the |
7884 | Lex-generated scanner to discard its current buffer and switch to the | |
7885 | new one. This depends upon your implementation of Lex; see its | |
7886 | documentation for more. For Flex, it suffices to call | |
7887 | @samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your | |
7888 | Flex-generated scanner needs to read from several input streams to | |
7889 | handle features like include files, you might consider using Flex | |
7890 | functions like @samp{yy_switch_to_buffer} that manipulate multiple | |
7891 | input buffers. | |
5b066063 | 7892 | |
b165c324 AD |
7893 | If your Flex-generated scanner uses start conditions (@pxref{Start |
7894 | conditions, , Start conditions, flex, The Flex Manual}), you might | |
7895 | also want to reset the scanner's state, i.e., go back to the initial | |
7896 | start condition, through a call to @samp{BEGIN (0)}. | |
7897 | ||
fef4cb51 AD |
7898 | @node Strings are Destroyed |
7899 | @section Strings are Destroyed | |
7900 | ||
7901 | @display | |
c7e441b4 | 7902 | My parser seems to destroy old strings, or maybe it loses track of |
fef4cb51 AD |
7903 | them. Instead of reporting @samp{"foo", "bar"}, it reports |
7904 | @samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}. | |
7905 | @end display | |
7906 | ||
7907 | This error is probably the single most frequent ``bug report'' sent to | |
7908 | Bison lists, but is only concerned with a misunderstanding of the role | |
8c5b881d | 7909 | of the scanner. Consider the following Lex code: |
fef4cb51 AD |
7910 | |
7911 | @verbatim | |
7912 | %{ | |
7913 | #include <stdio.h> | |
7914 | char *yylval = NULL; | |
7915 | %} | |
7916 | %% | |
7917 | .* yylval = yytext; return 1; | |
7918 | \n /* IGNORE */ | |
7919 | %% | |
7920 | int | |
7921 | main () | |
7922 | { | |
fa7e68c3 | 7923 | /* Similar to using $1, $2 in a Bison action. */ |
fef4cb51 AD |
7924 | char *fst = (yylex (), yylval); |
7925 | char *snd = (yylex (), yylval); | |
7926 | printf ("\"%s\", \"%s\"\n", fst, snd); | |
7927 | return 0; | |
7928 | } | |
7929 | @end verbatim | |
7930 | ||
7931 | If you compile and run this code, you get: | |
7932 | ||
7933 | @example | |
7934 | $ @kbd{flex -osplit-lines.c split-lines.l} | |
7935 | $ @kbd{gcc -osplit-lines split-lines.c -ll} | |
7936 | $ @kbd{printf 'one\ntwo\n' | ./split-lines} | |
7937 | "one | |
7938 | two", "two" | |
7939 | @end example | |
7940 | ||
7941 | @noindent | |
7942 | this is because @code{yytext} is a buffer provided for @emph{reading} | |
7943 | in the action, but if you want to keep it, you have to duplicate it | |
7944 | (e.g., using @code{strdup}). Note that the output may depend on how | |
7945 | your implementation of Lex handles @code{yytext}. For instance, when | |
7946 | given the Lex compatibility option @option{-l} (which triggers the | |
7947 | option @samp{%array}) Flex generates a different behavior: | |
7948 | ||
7949 | @example | |
7950 | $ @kbd{flex -l -osplit-lines.c split-lines.l} | |
7951 | $ @kbd{gcc -osplit-lines split-lines.c -ll} | |
7952 | $ @kbd{printf 'one\ntwo\n' | ./split-lines} | |
7953 | "two", "two" | |
7954 | @end example | |
7955 | ||
7956 | ||
2fa09258 AD |
7957 | @node Implementing Gotos/Loops |
7958 | @section Implementing Gotos/Loops | |
a06ea4aa AD |
7959 | |
7960 | @display | |
7961 | My simple calculator supports variables, assignments, and functions, | |
2fa09258 | 7962 | but how can I implement gotos, or loops? |
a06ea4aa AD |
7963 | @end display |
7964 | ||
7965 | Although very pedagogical, the examples included in the document blur | |
a1c84f45 | 7966 | the distinction to make between the parser---whose job is to recover |
a06ea4aa | 7967 | the structure of a text and to transmit it to subsequent modules of |
a1c84f45 | 7968 | the program---and the processing (such as the execution) of this |
a06ea4aa AD |
7969 | structure. This works well with so called straight line programs, |
7970 | i.e., precisely those that have a straightforward execution model: | |
7971 | execute simple instructions one after the others. | |
7972 | ||
7973 | @cindex abstract syntax tree | |
7974 | @cindex @acronym{AST} | |
7975 | If you want a richer model, you will probably need to use the parser | |
7976 | to construct a tree that does represent the structure it has | |
7977 | recovered; this tree is usually called the @dfn{abstract syntax tree}, | |
7978 | or @dfn{@acronym{AST}} for short. Then, walking through this tree, | |
7979 | traversing it in various ways, will enable treatments such as its | |
7980 | execution or its translation, which will result in an interpreter or a | |
7981 | compiler. | |
7982 | ||
7983 | This topic is way beyond the scope of this manual, and the reader is | |
7984 | invited to consult the dedicated literature. | |
7985 | ||
7986 | ||
ed2e6384 AD |
7987 | @node Multiple start-symbols |
7988 | @section Multiple start-symbols | |
7989 | ||
7990 | @display | |
7991 | I have several closely related grammars, and I would like to share their | |
7992 | implementations. In fact, I could use a single grammar but with | |
7993 | multiple entry points. | |
7994 | @end display | |
7995 | ||
7996 | Bison does not support multiple start-symbols, but there is a very | |
7997 | simple means to simulate them. If @code{foo} and @code{bar} are the two | |
7998 | pseudo start-symbols, then introduce two new tokens, say | |
7999 | @code{START_FOO} and @code{START_BAR}, and use them as switches from the | |
8000 | real start-symbol: | |
8001 | ||
8002 | @example | |
8003 | %token START_FOO START_BAR; | |
8004 | %start start; | |
8005 | start: START_FOO foo | |
8006 | | START_BAR bar; | |
8007 | @end example | |
8008 | ||
8009 | These tokens prevents the introduction of new conflicts. As far as the | |
8010 | parser goes, that is all that is needed. | |
8011 | ||
8012 | Now the difficult part is ensuring that the scanner will send these | |
8013 | tokens first. If your scanner is hand-written, that should be | |
8014 | straightforward. If your scanner is generated by Lex, them there is | |
8015 | simple means to do it: recall that anything between @samp{%@{ ... %@}} | |
8016 | after the first @code{%%} is copied verbatim in the top of the generated | |
8017 | @code{yylex} function. Make sure a variable @code{start_token} is | |
8018 | available in the scanner (e.g., a global variable or using | |
8019 | @code{%lex-param} etc.), and use the following: | |
8020 | ||
8021 | @example | |
8022 | /* @r{Prologue.} */ | |
8023 | %% | |
8024 | %@{ | |
8025 | if (start_token) | |
8026 | @{ | |
8027 | int t = start_token; | |
8028 | start_token = 0; | |
8029 | return t; | |
8030 | @} | |
8031 | %@} | |
8032 | /* @r{The rules.} */ | |
8033 | @end example | |
8034 | ||
8035 | ||
55ba27be AD |
8036 | @node Secure? Conform? |
8037 | @section Secure? Conform? | |
8038 | ||
8039 | @display | |
8040 | Is Bison secure? Does it conform to POSIX? | |
8041 | @end display | |
8042 | ||
8043 | If you're looking for a guarantee or certification, we don't provide it. | |
8044 | However, Bison is intended to be a reliable program that conforms to the | |
8045 | @acronym{POSIX} specification for Yacc. If you run into problems, | |
8046 | please send us a bug report. | |
8047 | ||
8048 | @node I can't build Bison | |
8049 | @section I can't build Bison | |
8050 | ||
8051 | @display | |
8c5b881d PE |
8052 | I can't build Bison because @command{make} complains that |
8053 | @code{msgfmt} is not found. | |
55ba27be AD |
8054 | What should I do? |
8055 | @end display | |
8056 | ||
8057 | Like most GNU packages with internationalization support, that feature | |
8058 | is turned on by default. If you have problems building in the @file{po} | |
8059 | subdirectory, it indicates that your system's internationalization | |
8060 | support is lacking. You can re-configure Bison with | |
8061 | @option{--disable-nls} to turn off this support, or you can install GNU | |
8062 | gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure | |
8063 | Bison. See the file @file{ABOUT-NLS} for more information. | |
8064 | ||
8065 | ||
8066 | @node Where can I find help? | |
8067 | @section Where can I find help? | |
8068 | ||
8069 | @display | |
8070 | I'm having trouble using Bison. Where can I find help? | |
8071 | @end display | |
8072 | ||
8073 | First, read this fine manual. Beyond that, you can send mail to | |
8074 | @email{help-bison@@gnu.org}. This mailing list is intended to be | |
8075 | populated with people who are willing to answer questions about using | |
8076 | and installing Bison. Please keep in mind that (most of) the people on | |
8077 | the list have aspects of their lives which are not related to Bison (!), | |
8078 | so you may not receive an answer to your question right away. This can | |
8079 | be frustrating, but please try not to honk them off; remember that any | |
8080 | help they provide is purely voluntary and out of the kindness of their | |
8081 | hearts. | |
8082 | ||
8083 | @node Bug Reports | |
8084 | @section Bug Reports | |
8085 | ||
8086 | @display | |
8087 | I found a bug. What should I include in the bug report? | |
8088 | @end display | |
8089 | ||
8090 | Before you send a bug report, make sure you are using the latest | |
8091 | version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its | |
8092 | mirrors. Be sure to include the version number in your bug report. If | |
8093 | the bug is present in the latest version but not in a previous version, | |
8094 | try to determine the most recent version which did not contain the bug. | |
8095 | ||
8096 | If the bug is parser-related, you should include the smallest grammar | |
8097 | you can which demonstrates the bug. The grammar file should also be | |
8098 | complete (i.e., I should be able to run it through Bison without having | |
8099 | to edit or add anything). The smaller and simpler the grammar, the | |
8100 | easier it will be to fix the bug. | |
8101 | ||
8102 | Include information about your compilation environment, including your | |
8103 | operating system's name and version and your compiler's name and | |
8104 | version. If you have trouble compiling, you should also include a | |
8105 | transcript of the build session, starting with the invocation of | |
8106 | `configure'. Depending on the nature of the bug, you may be asked to | |
8107 | send additional files as well (such as `config.h' or `config.cache'). | |
8108 | ||
8109 | Patches are most welcome, but not required. That is, do not hesitate to | |
8110 | send a bug report just because you can not provide a fix. | |
8111 | ||
8112 | Send bug reports to @email{bug-bison@@gnu.org}. | |
8113 | ||
8114 | @node Other Languages | |
8115 | @section Other Languages | |
8116 | ||
8117 | @display | |
8118 | Will Bison ever have C++ support? How about Java or @var{insert your | |
8119 | favorite language here}? | |
8120 | @end display | |
8121 | ||
8122 | C++ support is there now, and is documented. We'd love to add other | |
8123 | languages; contributions are welcome. | |
8124 | ||
8125 | @node Beta Testing | |
8126 | @section Beta Testing | |
8127 | ||
8128 | @display | |
8129 | What is involved in being a beta tester? | |
8130 | @end display | |
8131 | ||
8132 | It's not terribly involved. Basically, you would download a test | |
8133 | release, compile it, and use it to build and run a parser or two. After | |
8134 | that, you would submit either a bug report or a message saying that | |
8135 | everything is okay. It is important to report successes as well as | |
8136 | failures because test releases eventually become mainstream releases, | |
8137 | but only if they are adequately tested. If no one tests, development is | |
8138 | essentially halted. | |
8139 | ||
8140 | Beta testers are particularly needed for operating systems to which the | |
8141 | developers do not have easy access. They currently have easy access to | |
8142 | recent GNU/Linux and Solaris versions. Reports about other operating | |
8143 | systems are especially welcome. | |
8144 | ||
8145 | @node Mailing Lists | |
8146 | @section Mailing Lists | |
8147 | ||
8148 | @display | |
8149 | How do I join the help-bison and bug-bison mailing lists? | |
8150 | @end display | |
8151 | ||
8152 | See @url{http://lists.gnu.org/}. | |
a06ea4aa | 8153 | |
d1a1114f AD |
8154 | @c ================================================= Table of Symbols |
8155 | ||
342b8b6e | 8156 | @node Table of Symbols |
bfa74976 RS |
8157 | @appendix Bison Symbols |
8158 | @cindex Bison symbols, table of | |
8159 | @cindex symbols in Bison, table of | |
8160 | ||
18b519c0 | 8161 | @deffn {Variable} @@$ |
3ded9a63 | 8162 | In an action, the location of the left-hand side of the rule. |
88bce5a2 | 8163 | @xref{Locations, , Locations Overview}. |
18b519c0 | 8164 | @end deffn |
3ded9a63 | 8165 | |
18b519c0 | 8166 | @deffn {Variable} @@@var{n} |
3ded9a63 AD |
8167 | In an action, the location of the @var{n}-th symbol of the right-hand |
8168 | side of the rule. @xref{Locations, , Locations Overview}. | |
18b519c0 | 8169 | @end deffn |
3ded9a63 | 8170 | |
18b519c0 | 8171 | @deffn {Variable} $$ |
3ded9a63 AD |
8172 | In an action, the semantic value of the left-hand side of the rule. |
8173 | @xref{Actions}. | |
18b519c0 | 8174 | @end deffn |
3ded9a63 | 8175 | |
18b519c0 | 8176 | @deffn {Variable} $@var{n} |
3ded9a63 AD |
8177 | In an action, the semantic value of the @var{n}-th symbol of the |
8178 | right-hand side of the rule. @xref{Actions}. | |
18b519c0 | 8179 | @end deffn |
3ded9a63 | 8180 | |
dd8d9022 AD |
8181 | @deffn {Delimiter} %% |
8182 | Delimiter used to separate the grammar rule section from the | |
8183 | Bison declarations section or the epilogue. | |
8184 | @xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}. | |
18b519c0 | 8185 | @end deffn |
bfa74976 | 8186 | |
dd8d9022 AD |
8187 | @c Don't insert spaces, or check the DVI output. |
8188 | @deffn {Delimiter} %@{@var{code}%@} | |
8189 | All code listed between @samp{%@{} and @samp{%@}} is copied directly to | |
8190 | the output file uninterpreted. Such code forms the prologue of the input | |
8191 | file. @xref{Grammar Outline, ,Outline of a Bison | |
8192 | Grammar}. | |
18b519c0 | 8193 | @end deffn |
bfa74976 | 8194 | |
dd8d9022 AD |
8195 | @deffn {Construct} /*@dots{}*/ |
8196 | Comment delimiters, as in C. | |
18b519c0 | 8197 | @end deffn |
bfa74976 | 8198 | |
dd8d9022 AD |
8199 | @deffn {Delimiter} : |
8200 | Separates a rule's result from its components. @xref{Rules, ,Syntax of | |
8201 | Grammar Rules}. | |
18b519c0 | 8202 | @end deffn |
bfa74976 | 8203 | |
dd8d9022 AD |
8204 | @deffn {Delimiter} ; |
8205 | Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}. | |
18b519c0 | 8206 | @end deffn |
bfa74976 | 8207 | |
dd8d9022 AD |
8208 | @deffn {Delimiter} | |
8209 | Separates alternate rules for the same result nonterminal. | |
8210 | @xref{Rules, ,Syntax of Grammar Rules}. | |
18b519c0 | 8211 | @end deffn |
bfa74976 | 8212 | |
dd8d9022 AD |
8213 | @deffn {Symbol} $accept |
8214 | The predefined nonterminal whose only rule is @samp{$accept: @var{start} | |
8215 | $end}, where @var{start} is the start symbol. @xref{Start Decl, , The | |
8216 | Start-Symbol}. It cannot be used in the grammar. | |
18b519c0 | 8217 | @end deffn |
bfa74976 | 8218 | |
34f98f46 JD |
8219 | @deffn {Directive} %after-header @{@var{code}@} |
8220 | Specifies code to be inserted into the code file after the contents of the | |
8221 | header file. | |
8222 | @xref{Table of Symbols, ,%start-header}. | |
8223 | @end deffn | |
8224 | ||
8225 | @deffn {Directive} %before-header @{@var{code}@} | |
8226 | Specifies code to be inserted into the code file before the contents of the | |
8227 | header file. | |
8228 | @xref{Table of Symbols, ,%start-header}. | |
8229 | @end deffn | |
8230 | ||
8231 | @deffn {Directive} %end-header @{@var{code}@} | |
9bc0dd67 JD |
8232 | Specifies code to be inserted both into the header file (if generated; |
8233 | @pxref{Table of Symbols, ,%defines}) and into the code file after any | |
8234 | Bison-generated definitions. | |
34f98f46 | 8235 | @xref{Table of Symbols, ,%start-header}. |
9bc0dd67 JD |
8236 | @end deffn |
8237 | ||
34f98f46 | 8238 | @deffn {Directive} %start-header @{@var{code}@} |
9bc0dd67 JD |
8239 | Specifies code to be inserted both into the header file (if generated; |
8240 | @pxref{Table of Symbols, ,%defines}) and into the code file before any | |
8241 | Bison-generated definitions. | |
8242 | ||
8243 | @cindex Prologue | |
34f98f46 | 8244 | @findex %before-header |
9bc0dd67 | 8245 | @findex %union |
34f98f46 JD |
8246 | @findex %end-header |
8247 | @findex %after-header | |
8248 | For example, the following declaration order in the grammar file reflects the | |
8249 | order in which Bison will output these code blocks. However, you are free to | |
8250 | declare these code blocks in your grammar file in whatever order is most | |
8251 | convenient for you: | |
9bc0dd67 JD |
8252 | |
8253 | @smallexample | |
34f98f46 JD |
8254 | %before-header @{ |
8255 | /* Bison treats this block like a pre-prologue block: it inserts it | |
8256 | * into the code file before the contents of the header file. It | |
8257 | * does *not* insert it into the header file. This is a good place | |
8258 | * to put #include's that you want at the top of your code file. A | |
8259 | * common example is `#include "system.h"'. */ | |
8260 | @} | |
8261 | %start-header @{ | |
8262 | /* Bison inserts this block into both the header file and the code | |
8263 | * file. In both files, the point of insertion is before any | |
8264 | * Bison-generated token, semantic type, location type, and class | |
8265 | * definitions. This is a good place to define %union | |
8266 | * dependencies, for example. */ | |
9bc0dd67 JD |
8267 | @} |
8268 | %union @{ | |
34f98f46 JD |
8269 | /* Unlike the traditional Yacc prologue blocks, the output order |
8270 | * for the %*-header blocks is not affected by their declaration | |
8271 | * position relative to any %union in the grammar file. */ | |
9bc0dd67 | 8272 | @} |
34f98f46 JD |
8273 | %end-header @{ |
8274 | /* Bison inserts this block into both the header file and the code | |
8275 | * file. In both files, the point of insertion is after the | |
8276 | * Bison-generated definitions. This is a good place to declare or | |
8277 | * define public functions or data structures that depend on the | |
8278 | * Bison-generated definitions. */ | |
8279 | @} | |
8280 | %after-header @{ | |
8281 | /* Bison treats this block like a post-prologue block: it inserts | |
8282 | * it into the code file after the contents of the header file. It | |
8283 | * does *not* insert it into the header file. This is a good place | |
8284 | * to declare or define internal functions or data structures that | |
8285 | * depend on the Bison-generated definitions. */ | |
9bc0dd67 | 8286 | @} |
9bc0dd67 JD |
8287 | @end smallexample |
8288 | ||
34f98f46 JD |
8289 | If you have multiple occurrences of any one of the above declarations, Bison |
8290 | will concatenate the contents in declaration order. | |
8291 | ||
9bc0dd67 JD |
8292 | @xref{Prologue, ,The Prologue}. |
8293 | @end deffn | |
8294 | ||
8295 | @deffn {Directive} %debug | |
8296 | Equip the parser for debugging. @xref{Decl Summary}. | |
8297 | @end deffn | |
8298 | ||
18b519c0 | 8299 | @deffn {Directive} %debug |
6deb4447 | 8300 | Equip the parser for debugging. @xref{Decl Summary}. |
18b519c0 | 8301 | @end deffn |
6deb4447 | 8302 | |
91d2c560 | 8303 | @ifset defaultprec |
22fccf95 PE |
8304 | @deffn {Directive} %default-prec |
8305 | Assign a precedence to rules that lack an explicit @samp{%prec} | |
8306 | modifier. @xref{Contextual Precedence, ,Context-Dependent | |
8307 | Precedence}. | |
39a06c25 | 8308 | @end deffn |
91d2c560 | 8309 | @end ifset |
39a06c25 | 8310 | |
18b519c0 | 8311 | @deffn {Directive} %defines |
6deb4447 AD |
8312 | Bison declaration to create a header file meant for the scanner. |
8313 | @xref{Decl Summary}. | |
18b519c0 | 8314 | @end deffn |
6deb4447 | 8315 | |
18b519c0 | 8316 | @deffn {Directive} %destructor |
258b75ca | 8317 | Specify how the parser should reclaim the memory associated to |
fa7e68c3 | 8318 | discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}. |
18b519c0 | 8319 | @end deffn |
72f889cc | 8320 | |
18b519c0 | 8321 | @deffn {Directive} %dprec |
676385e2 | 8322 | Bison declaration to assign a precedence to a rule that is used at parse |
c827f760 PE |
8323 | time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing |
8324 | @acronym{GLR} Parsers}. | |
18b519c0 | 8325 | @end deffn |
676385e2 | 8326 | |
dd8d9022 AD |
8327 | @deffn {Symbol} $end |
8328 | The predefined token marking the end of the token stream. It cannot be | |
8329 | used in the grammar. | |
8330 | @end deffn | |
8331 | ||
8332 | @deffn {Symbol} error | |
8333 | A token name reserved for error recovery. This token may be used in | |
8334 | grammar rules so as to allow the Bison parser to recognize an error in | |
8335 | the grammar without halting the process. In effect, a sentence | |
8336 | containing an error may be recognized as valid. On a syntax error, the | |
742e4900 JD |
8337 | token @code{error} becomes the current lookahead token. Actions |
8338 | corresponding to @code{error} are then executed, and the lookahead | |
dd8d9022 AD |
8339 | token is reset to the token that originally caused the violation. |
8340 | @xref{Error Recovery}. | |
18d192f0 AD |
8341 | @end deffn |
8342 | ||
18b519c0 | 8343 | @deffn {Directive} %error-verbose |
2a8d363a AD |
8344 | Bison declaration to request verbose, specific error message strings |
8345 | when @code{yyerror} is called. | |
18b519c0 | 8346 | @end deffn |
2a8d363a | 8347 | |
18b519c0 | 8348 | @deffn {Directive} %file-prefix="@var{prefix}" |
72d2299c | 8349 | Bison declaration to set the prefix of the output files. @xref{Decl |
d8988b2f | 8350 | Summary}. |
18b519c0 | 8351 | @end deffn |
d8988b2f | 8352 | |
18b519c0 | 8353 | @deffn {Directive} %glr-parser |
c827f760 PE |
8354 | Bison declaration to produce a @acronym{GLR} parser. @xref{GLR |
8355 | Parsers, ,Writing @acronym{GLR} Parsers}. | |
18b519c0 | 8356 | @end deffn |
676385e2 | 8357 | |
dd8d9022 AD |
8358 | @deffn {Directive} %initial-action |
8359 | Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}. | |
8360 | @end deffn | |
8361 | ||
18b519c0 | 8362 | @deffn {Directive} %left |
bfa74976 RS |
8363 | Bison declaration to assign left associativity to token(s). |
8364 | @xref{Precedence Decl, ,Operator Precedence}. | |
18b519c0 | 8365 | @end deffn |
bfa74976 | 8366 | |
feeb0eda | 8367 | @deffn {Directive} %lex-param @{@var{argument-declaration}@} |
2a8d363a AD |
8368 | Bison declaration to specifying an additional parameter that |
8369 | @code{yylex} should accept. @xref{Pure Calling,, Calling Conventions | |
8370 | for Pure Parsers}. | |
18b519c0 | 8371 | @end deffn |
2a8d363a | 8372 | |
18b519c0 | 8373 | @deffn {Directive} %merge |
676385e2 | 8374 | Bison declaration to assign a merging function to a rule. If there is a |
fae437e8 | 8375 | reduce/reduce conflict with a rule having the same merging function, the |
676385e2 | 8376 | function is applied to the two semantic values to get a single result. |
c827f760 | 8377 | @xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}. |
18b519c0 | 8378 | @end deffn |
676385e2 | 8379 | |
18b519c0 | 8380 | @deffn {Directive} %name-prefix="@var{prefix}" |
72d2299c | 8381 | Bison declaration to rename the external symbols. @xref{Decl Summary}. |
18b519c0 | 8382 | @end deffn |
d8988b2f | 8383 | |
91d2c560 | 8384 | @ifset defaultprec |
22fccf95 PE |
8385 | @deffn {Directive} %no-default-prec |
8386 | Do not assign a precedence to rules that lack an explicit @samp{%prec} | |
8387 | modifier. @xref{Contextual Precedence, ,Context-Dependent | |
8388 | Precedence}. | |
8389 | @end deffn | |
91d2c560 | 8390 | @end ifset |
22fccf95 | 8391 | |
18b519c0 | 8392 | @deffn {Directive} %no-lines |
931c7513 RS |
8393 | Bison declaration to avoid generating @code{#line} directives in the |
8394 | parser file. @xref{Decl Summary}. | |
18b519c0 | 8395 | @end deffn |
931c7513 | 8396 | |
18b519c0 | 8397 | @deffn {Directive} %nonassoc |
9d9b8b70 | 8398 | Bison declaration to assign nonassociativity to token(s). |
bfa74976 | 8399 | @xref{Precedence Decl, ,Operator Precedence}. |
18b519c0 | 8400 | @end deffn |
bfa74976 | 8401 | |
fa4d969f | 8402 | @deffn {Directive} %output="@var{file}" |
72d2299c | 8403 | Bison declaration to set the name of the parser file. @xref{Decl |
d8988b2f | 8404 | Summary}. |
18b519c0 | 8405 | @end deffn |
d8988b2f | 8406 | |
feeb0eda | 8407 | @deffn {Directive} %parse-param @{@var{argument-declaration}@} |
2a8d363a AD |
8408 | Bison declaration to specifying an additional parameter that |
8409 | @code{yyparse} should accept. @xref{Parser Function,, The Parser | |
8410 | Function @code{yyparse}}. | |
18b519c0 | 8411 | @end deffn |
2a8d363a | 8412 | |
18b519c0 | 8413 | @deffn {Directive} %prec |
bfa74976 RS |
8414 | Bison declaration to assign a precedence to a specific rule. |
8415 | @xref{Contextual Precedence, ,Context-Dependent Precedence}. | |
18b519c0 | 8416 | @end deffn |
bfa74976 | 8417 | |
18b519c0 | 8418 | @deffn {Directive} %pure-parser |
bfa74976 RS |
8419 | Bison declaration to request a pure (reentrant) parser. |
8420 | @xref{Pure Decl, ,A Pure (Reentrant) Parser}. | |
18b519c0 | 8421 | @end deffn |
bfa74976 | 8422 | |
b50d2359 | 8423 | @deffn {Directive} %require "@var{version}" |
9b8a5ce0 AD |
8424 | Require version @var{version} or higher of Bison. @xref{Require Decl, , |
8425 | Require a Version of Bison}. | |
b50d2359 AD |
8426 | @end deffn |
8427 | ||
18b519c0 | 8428 | @deffn {Directive} %right |
bfa74976 RS |
8429 | Bison declaration to assign right associativity to token(s). |
8430 | @xref{Precedence Decl, ,Operator Precedence}. | |
18b519c0 | 8431 | @end deffn |
bfa74976 | 8432 | |
18b519c0 | 8433 | @deffn {Directive} %start |
704a47c4 AD |
8434 | Bison declaration to specify the start symbol. @xref{Start Decl, ,The |
8435 | Start-Symbol}. | |
18b519c0 | 8436 | @end deffn |
bfa74976 | 8437 | |
18b519c0 | 8438 | @deffn {Directive} %token |
bfa74976 RS |
8439 | Bison declaration to declare token(s) without specifying precedence. |
8440 | @xref{Token Decl, ,Token Type Names}. | |
18b519c0 | 8441 | @end deffn |
bfa74976 | 8442 | |
18b519c0 | 8443 | @deffn {Directive} %token-table |
931c7513 RS |
8444 | Bison declaration to include a token name table in the parser file. |
8445 | @xref{Decl Summary}. | |
18b519c0 | 8446 | @end deffn |
931c7513 | 8447 | |
18b519c0 | 8448 | @deffn {Directive} %type |
704a47c4 AD |
8449 | Bison declaration to declare nonterminals. @xref{Type Decl, |
8450 | ,Nonterminal Symbols}. | |
18b519c0 | 8451 | @end deffn |
bfa74976 | 8452 | |
dd8d9022 AD |
8453 | @deffn {Symbol} $undefined |
8454 | The predefined token onto which all undefined values returned by | |
8455 | @code{yylex} are mapped. It cannot be used in the grammar, rather, use | |
8456 | @code{error}. | |
8457 | @end deffn | |
8458 | ||
18b519c0 | 8459 | @deffn {Directive} %union |
bfa74976 RS |
8460 | Bison declaration to specify several possible data types for semantic |
8461 | values. @xref{Union Decl, ,The Collection of Value Types}. | |
18b519c0 | 8462 | @end deffn |
bfa74976 | 8463 | |
dd8d9022 AD |
8464 | @deffn {Macro} YYABORT |
8465 | Macro to pretend that an unrecoverable syntax error has occurred, by | |
8466 | making @code{yyparse} return 1 immediately. The error reporting | |
8467 | function @code{yyerror} is not called. @xref{Parser Function, ,The | |
8468 | Parser Function @code{yyparse}}. | |
8469 | @end deffn | |
3ded9a63 | 8470 | |
dd8d9022 AD |
8471 | @deffn {Macro} YYACCEPT |
8472 | Macro to pretend that a complete utterance of the language has been | |
8473 | read, by making @code{yyparse} return 0 immediately. | |
8474 | @xref{Parser Function, ,The Parser Function @code{yyparse}}. | |
8475 | @end deffn | |
bfa74976 | 8476 | |
dd8d9022 | 8477 | @deffn {Macro} YYBACKUP |
742e4900 | 8478 | Macro to discard a value from the parser stack and fake a lookahead |
dd8d9022 | 8479 | token. @xref{Action Features, ,Special Features for Use in Actions}. |
18b519c0 | 8480 | @end deffn |
bfa74976 | 8481 | |
dd8d9022 | 8482 | @deffn {Variable} yychar |
32c29292 | 8483 | External integer variable that contains the integer value of the |
742e4900 | 8484 | lookahead token. (In a pure parser, it is a local variable within |
dd8d9022 AD |
8485 | @code{yyparse}.) Error-recovery rule actions may examine this variable. |
8486 | @xref{Action Features, ,Special Features for Use in Actions}. | |
18b519c0 | 8487 | @end deffn |
bfa74976 | 8488 | |
dd8d9022 AD |
8489 | @deffn {Variable} yyclearin |
8490 | Macro used in error-recovery rule actions. It clears the previous | |
742e4900 | 8491 | lookahead token. @xref{Error Recovery}. |
18b519c0 | 8492 | @end deffn |
bfa74976 | 8493 | |
dd8d9022 AD |
8494 | @deffn {Macro} YYDEBUG |
8495 | Macro to define to equip the parser with tracing code. @xref{Tracing, | |
8496 | ,Tracing Your Parser}. | |
18b519c0 | 8497 | @end deffn |
bfa74976 | 8498 | |
dd8d9022 AD |
8499 | @deffn {Variable} yydebug |
8500 | External integer variable set to zero by default. If @code{yydebug} | |
8501 | is given a nonzero value, the parser will output information on input | |
8502 | symbols and parser action. @xref{Tracing, ,Tracing Your Parser}. | |
18b519c0 | 8503 | @end deffn |
bfa74976 | 8504 | |
dd8d9022 AD |
8505 | @deffn {Macro} yyerrok |
8506 | Macro to cause parser to recover immediately to its normal mode | |
8507 | after a syntax error. @xref{Error Recovery}. | |
8508 | @end deffn | |
8509 | ||
8510 | @deffn {Macro} YYERROR | |
8511 | Macro to pretend that a syntax error has just been detected: call | |
8512 | @code{yyerror} and then perform normal error recovery if possible | |
8513 | (@pxref{Error Recovery}), or (if recovery is impossible) make | |
8514 | @code{yyparse} return 1. @xref{Error Recovery}. | |
8515 | @end deffn | |
8516 | ||
8517 | @deffn {Function} yyerror | |
8518 | User-supplied function to be called by @code{yyparse} on error. | |
8519 | @xref{Error Reporting, ,The Error | |
8520 | Reporting Function @code{yyerror}}. | |
8521 | @end deffn | |
8522 | ||
8523 | @deffn {Macro} YYERROR_VERBOSE | |
8524 | An obsolete macro that you define with @code{#define} in the prologue | |
8525 | to request verbose, specific error message strings | |
8526 | when @code{yyerror} is called. It doesn't matter what definition you | |
8527 | use for @code{YYERROR_VERBOSE}, just whether you define it. Using | |
8528 | @code{%error-verbose} is preferred. | |
8529 | @end deffn | |
8530 | ||
8531 | @deffn {Macro} YYINITDEPTH | |
8532 | Macro for specifying the initial size of the parser stack. | |
1a059451 | 8533 | @xref{Memory Management}. |
dd8d9022 AD |
8534 | @end deffn |
8535 | ||
8536 | @deffn {Function} yylex | |
8537 | User-supplied lexical analyzer function, called with no arguments to get | |
8538 | the next token. @xref{Lexical, ,The Lexical Analyzer Function | |
8539 | @code{yylex}}. | |
8540 | @end deffn | |
8541 | ||
8542 | @deffn {Macro} YYLEX_PARAM | |
8543 | An obsolete macro for specifying an extra argument (or list of extra | |
32c29292 | 8544 | arguments) for @code{yyparse} to pass to @code{yylex}. The use of this |
dd8d9022 AD |
8545 | macro is deprecated, and is supported only for Yacc like parsers. |
8546 | @xref{Pure Calling,, Calling Conventions for Pure Parsers}. | |
8547 | @end deffn | |
8548 | ||
8549 | @deffn {Variable} yylloc | |
8550 | External variable in which @code{yylex} should place the line and column | |
8551 | numbers associated with a token. (In a pure parser, it is a local | |
8552 | variable within @code{yyparse}, and its address is passed to | |
32c29292 JD |
8553 | @code{yylex}.) |
8554 | You can ignore this variable if you don't use the @samp{@@} feature in the | |
8555 | grammar actions. | |
8556 | @xref{Token Locations, ,Textual Locations of Tokens}. | |
742e4900 | 8557 | In semantic actions, it stores the location of the lookahead token. |
32c29292 | 8558 | @xref{Actions and Locations, ,Actions and Locations}. |
dd8d9022 AD |
8559 | @end deffn |
8560 | ||
8561 | @deffn {Type} YYLTYPE | |
8562 | Data type of @code{yylloc}; by default, a structure with four | |
8563 | members. @xref{Location Type, , Data Types of Locations}. | |
8564 | @end deffn | |
8565 | ||
8566 | @deffn {Variable} yylval | |
8567 | External variable in which @code{yylex} should place the semantic | |
8568 | value associated with a token. (In a pure parser, it is a local | |
8569 | variable within @code{yyparse}, and its address is passed to | |
32c29292 JD |
8570 | @code{yylex}.) |
8571 | @xref{Token Values, ,Semantic Values of Tokens}. | |
742e4900 | 8572 | In semantic actions, it stores the semantic value of the lookahead token. |
32c29292 | 8573 | @xref{Actions, ,Actions}. |
dd8d9022 AD |
8574 | @end deffn |
8575 | ||
8576 | @deffn {Macro} YYMAXDEPTH | |
1a059451 PE |
8577 | Macro for specifying the maximum size of the parser stack. @xref{Memory |
8578 | Management}. | |
dd8d9022 AD |
8579 | @end deffn |
8580 | ||
8581 | @deffn {Variable} yynerrs | |
8a2800e7 | 8582 | Global variable which Bison increments each time it reports a syntax error. |
dd8d9022 AD |
8583 | (In a pure parser, it is a local variable within @code{yyparse}.) |
8584 | @xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}. | |
8585 | @end deffn | |
8586 | ||
8587 | @deffn {Function} yyparse | |
8588 | The parser function produced by Bison; call this function to start | |
8589 | parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}. | |
8590 | @end deffn | |
8591 | ||
8592 | @deffn {Macro} YYPARSE_PARAM | |
8593 | An obsolete macro for specifying the name of a parameter that | |
8594 | @code{yyparse} should accept. The use of this macro is deprecated, and | |
8595 | is supported only for Yacc like parsers. @xref{Pure Calling,, Calling | |
8596 | Conventions for Pure Parsers}. | |
8597 | @end deffn | |
8598 | ||
8599 | @deffn {Macro} YYRECOVERING | |
02103984 PE |
8600 | The expression @code{YYRECOVERING ()} yields 1 when the parser |
8601 | is recovering from a syntax error, and 0 otherwise. | |
8602 | @xref{Action Features, ,Special Features for Use in Actions}. | |
dd8d9022 AD |
8603 | @end deffn |
8604 | ||
8605 | @deffn {Macro} YYSTACK_USE_ALLOCA | |
d7e14fc0 PE |
8606 | Macro used to control the use of @code{alloca} when the C |
8607 | @acronym{LALR}(1) parser needs to extend its stacks. If defined to 0, | |
8608 | the parser will use @code{malloc} to extend its stacks. If defined to | |
8609 | 1, the parser will use @code{alloca}. Values other than 0 and 1 are | |
8610 | reserved for future Bison extensions. If not defined, | |
8611 | @code{YYSTACK_USE_ALLOCA} defaults to 0. | |
8612 | ||
55289366 | 8613 | In the all-too-common case where your code may run on a host with a |
d7e14fc0 PE |
8614 | limited stack and with unreliable stack-overflow checking, you should |
8615 | set @code{YYMAXDEPTH} to a value that cannot possibly result in | |
8616 | unchecked stack overflow on any of your target hosts when | |
8617 | @code{alloca} is called. You can inspect the code that Bison | |
8618 | generates in order to determine the proper numeric values. This will | |
8619 | require some expertise in low-level implementation details. | |
dd8d9022 AD |
8620 | @end deffn |
8621 | ||
8622 | @deffn {Type} YYSTYPE | |
8623 | Data type of semantic values; @code{int} by default. | |
8624 | @xref{Value Type, ,Data Types of Semantic Values}. | |
18b519c0 | 8625 | @end deffn |
bfa74976 | 8626 | |
342b8b6e | 8627 | @node Glossary |
bfa74976 RS |
8628 | @appendix Glossary |
8629 | @cindex glossary | |
8630 | ||
8631 | @table @asis | |
c827f760 PE |
8632 | @item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'') |
8633 | Formal method of specifying context-free grammars originally proposed | |
8634 | by John Backus, and slightly improved by Peter Naur in his 1960-01-02 | |
8635 | committee document contributing to what became the Algol 60 report. | |
8636 | @xref{Language and Grammar, ,Languages and Context-Free Grammars}. | |
bfa74976 RS |
8637 | |
8638 | @item Context-free grammars | |
8639 | Grammars specified as rules that can be applied regardless of context. | |
8640 | Thus, if there is a rule which says that an integer can be used as an | |
8641 | expression, integers are allowed @emph{anywhere} an expression is | |
89cab50d AD |
8642 | permitted. @xref{Language and Grammar, ,Languages and Context-Free |
8643 | Grammars}. | |
bfa74976 RS |
8644 | |
8645 | @item Dynamic allocation | |
8646 | Allocation of memory that occurs during execution, rather than at | |
8647 | compile time or on entry to a function. | |
8648 | ||
8649 | @item Empty string | |
8650 | Analogous to the empty set in set theory, the empty string is a | |
8651 | character string of length zero. | |
8652 | ||
8653 | @item Finite-state stack machine | |
8654 | A ``machine'' that has discrete states in which it is said to exist at | |
8655 | each instant in time. As input to the machine is processed, the | |
8656 | machine moves from state to state as specified by the logic of the | |
8657 | machine. In the case of the parser, the input is the language being | |
8658 | parsed, and the states correspond to various stages in the grammar | |
c827f760 | 8659 | rules. @xref{Algorithm, ,The Bison Parser Algorithm}. |
bfa74976 | 8660 | |
c827f760 | 8661 | @item Generalized @acronym{LR} (@acronym{GLR}) |
676385e2 | 8662 | A parsing algorithm that can handle all context-free grammars, including those |
c827f760 PE |
8663 | that are not @acronym{LALR}(1). It resolves situations that Bison's |
8664 | usual @acronym{LALR}(1) | |
676385e2 PH |
8665 | algorithm cannot by effectively splitting off multiple parsers, trying all |
8666 | possible parsers, and discarding those that fail in the light of additional | |
c827f760 PE |
8667 | right context. @xref{Generalized LR Parsing, ,Generalized |
8668 | @acronym{LR} Parsing}. | |
676385e2 | 8669 | |
bfa74976 RS |
8670 | @item Grouping |
8671 | A language construct that is (in general) grammatically divisible; | |
c827f760 | 8672 | for example, `expression' or `declaration' in C@. |
bfa74976 RS |
8673 | @xref{Language and Grammar, ,Languages and Context-Free Grammars}. |
8674 | ||
8675 | @item Infix operator | |
8676 | An arithmetic operator that is placed between the operands on which it | |
8677 | performs some operation. | |
8678 | ||
8679 | @item Input stream | |
8680 | A continuous flow of data between devices or programs. | |
8681 | ||
8682 | @item Language construct | |
8683 | One of the typical usage schemas of the language. For example, one of | |
8684 | the constructs of the C language is the @code{if} statement. | |
8685 | @xref{Language and Grammar, ,Languages and Context-Free Grammars}. | |
8686 | ||
8687 | @item Left associativity | |
8688 | Operators having left associativity are analyzed from left to right: | |
8689 | @samp{a+b+c} first computes @samp{a+b} and then combines with | |
8690 | @samp{c}. @xref{Precedence, ,Operator Precedence}. | |
8691 | ||
8692 | @item Left recursion | |
89cab50d AD |
8693 | A rule whose result symbol is also its first component symbol; for |
8694 | example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive | |
8695 | Rules}. | |
bfa74976 RS |
8696 | |
8697 | @item Left-to-right parsing | |
8698 | Parsing a sentence of a language by analyzing it token by token from | |
c827f760 | 8699 | left to right. @xref{Algorithm, ,The Bison Parser Algorithm}. |
bfa74976 RS |
8700 | |
8701 | @item Lexical analyzer (scanner) | |
8702 | A function that reads an input stream and returns tokens one by one. | |
8703 | @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}. | |
8704 | ||
8705 | @item Lexical tie-in | |
8706 | A flag, set by actions in the grammar rules, which alters the way | |
8707 | tokens are parsed. @xref{Lexical Tie-ins}. | |
8708 | ||
931c7513 | 8709 | @item Literal string token |
14ded682 | 8710 | A token which consists of two or more fixed characters. @xref{Symbols}. |
931c7513 | 8711 | |
742e4900 JD |
8712 | @item Lookahead token |
8713 | A token already read but not yet shifted. @xref{Lookahead, ,Lookahead | |
89cab50d | 8714 | Tokens}. |
bfa74976 | 8715 | |
c827f760 | 8716 | @item @acronym{LALR}(1) |
bfa74976 | 8717 | The class of context-free grammars that Bison (like most other parser |
c827f760 PE |
8718 | generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery |
8719 | Conflicts, ,Mysterious Reduce/Reduce Conflicts}. | |
bfa74976 | 8720 | |
c827f760 | 8721 | @item @acronym{LR}(1) |
bfa74976 | 8722 | The class of context-free grammars in which at most one token of |
742e4900 | 8723 | lookahead is needed to disambiguate the parsing of any piece of input. |
bfa74976 RS |
8724 | |
8725 | @item Nonterminal symbol | |
8726 | A grammar symbol standing for a grammatical construct that can | |
8727 | be expressed through rules in terms of smaller constructs; in other | |
8728 | words, a construct that is not a token. @xref{Symbols}. | |
8729 | ||
bfa74976 RS |
8730 | @item Parser |
8731 | A function that recognizes valid sentences of a language by analyzing | |
8732 | the syntax structure of a set of tokens passed to it from a lexical | |
8733 | analyzer. | |
8734 | ||
8735 | @item Postfix operator | |
8736 | An arithmetic operator that is placed after the operands upon which it | |
8737 | performs some operation. | |
8738 | ||
8739 | @item Reduction | |
8740 | Replacing a string of nonterminals and/or terminals with a single | |
89cab50d | 8741 | nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison |
c827f760 | 8742 | Parser Algorithm}. |
bfa74976 RS |
8743 | |
8744 | @item Reentrant | |
8745 | A reentrant subprogram is a subprogram which can be in invoked any | |
8746 | number of times in parallel, without interference between the various | |
8747 | invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}. | |
8748 | ||
8749 | @item Reverse polish notation | |
8750 | A language in which all operators are postfix operators. | |
8751 | ||
8752 | @item Right recursion | |
89cab50d AD |
8753 | A rule whose result symbol is also its last component symbol; for |
8754 | example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive | |
8755 | Rules}. | |
bfa74976 RS |
8756 | |
8757 | @item Semantics | |
8758 | In computer languages, the semantics are specified by the actions | |
8759 | taken for each instance of the language, i.e., the meaning of | |
8760 | each statement. @xref{Semantics, ,Defining Language Semantics}. | |
8761 | ||
8762 | @item Shift | |
8763 | A parser is said to shift when it makes the choice of analyzing | |
8764 | further input from the stream rather than reducing immediately some | |
c827f760 | 8765 | already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}. |
bfa74976 RS |
8766 | |
8767 | @item Single-character literal | |
8768 | A single character that is recognized and interpreted as is. | |
8769 | @xref{Grammar in Bison, ,From Formal Rules to Bison Input}. | |
8770 | ||
8771 | @item Start symbol | |
8772 | The nonterminal symbol that stands for a complete valid utterance in | |
8773 | the language being parsed. The start symbol is usually listed as the | |
13863333 | 8774 | first nonterminal symbol in a language specification. |
bfa74976 RS |
8775 | @xref{Start Decl, ,The Start-Symbol}. |
8776 | ||
8777 | @item Symbol table | |
8778 | A data structure where symbol names and associated data are stored | |
8779 | during parsing to allow for recognition and use of existing | |
8780 | information in repeated uses of a symbol. @xref{Multi-function Calc}. | |
8781 | ||
6e649e65 PE |
8782 | @item Syntax error |
8783 | An error encountered during parsing of an input stream due to invalid | |
8784 | syntax. @xref{Error Recovery}. | |
8785 | ||
bfa74976 RS |
8786 | @item Token |
8787 | A basic, grammatically indivisible unit of a language. The symbol | |
8788 | that describes a token in the grammar is a terminal symbol. | |
8789 | The input of the Bison parser is a stream of tokens which comes from | |
8790 | the lexical analyzer. @xref{Symbols}. | |
8791 | ||
8792 | @item Terminal symbol | |
89cab50d AD |
8793 | A grammar symbol that has no rules in the grammar and therefore is |
8794 | grammatically indivisible. The piece of text it represents is a token. | |
8795 | @xref{Language and Grammar, ,Languages and Context-Free Grammars}. | |
bfa74976 RS |
8796 | @end table |
8797 | ||
342b8b6e | 8798 | @node Copying This Manual |
f2b5126e | 8799 | @appendix Copying This Manual |
f9a8293a | 8800 | |
f2b5126e PB |
8801 | @menu |
8802 | * GNU Free Documentation License:: License for copying this manual. | |
8803 | @end menu | |
f9a8293a | 8804 | |
f2b5126e PB |
8805 | @include fdl.texi |
8806 | ||
342b8b6e | 8807 | @node Index |
bfa74976 RS |
8808 | @unnumbered Index |
8809 | ||
8810 | @printindex cp | |
8811 | ||
bfa74976 | 8812 | @bye |
a06ea4aa AD |
8813 | |
8814 | @c LocalWords: texinfo setfilename settitle setchapternewpage finalout | |
8815 | @c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex | |
8816 | @c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry | |
8817 | @c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa | |
8818 | @c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc | |
8819 | @c LocalWords: rpcalc Lexer Gen Comp Expr ltcalc mfcalc Decl Symtab yylex | |
8820 | @c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref | |
8821 | @c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex | |
8822 | @c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge | |
8823 | @c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG | |
8824 | @c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit | |
8825 | @c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok | |
8826 | @c LocalWords: longjmp fprintf stderr preg yylloc YYLTYPE cos ln | |
8827 | @c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym | |
8828 | @c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof | |
8829 | @c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum | |
8830 | @c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype | |
8831 | @c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless typefull yynerrs | |
8832 | @c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES | |
8833 | @c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param | |
8834 | @c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP | |
32c29292 | 8835 | @c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword |
a06ea4aa AD |
8836 | @c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH |
8837 | @c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm VCG notype | |
8838 | @c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args | |
8839 | @c LocalWords: YYPRINTF infile ypp yxx outfile itemx vcg tex leaderfill | |
8840 | @c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll | |
8841 | @c LocalWords: yyrestart nbar yytext fst snd osplit ntwo strdup AST | |
8842 | @c LocalWords: YYSTACK DVI fdl printindex |