]>
Commit | Line | Data |
---|---|---|
bfa74976 RS |
1 | \input texinfo @c -*-texinfo-*- |
2 | @comment %**start of header | |
3 | @setfilename bison.info | |
df1af54c JT |
4 | @include version.texi |
5 | @settitle Bison @value{VERSION} | |
bfa74976 RS |
6 | @setchapternewpage odd |
7 | ||
5378c3e7 DM |
8 | @iftex |
9 | @finalout | |
10 | @end iftex | |
11 | ||
13863333 | 12 | @c SMALL BOOK version |
bfa74976 | 13 | @c This edition has been formatted so that you can format and print it in |
13863333 | 14 | @c the smallbook format. |
bfa74976 RS |
15 | @c @smallbook |
16 | ||
bfa74976 RS |
17 | @c Set following if you have the new `shorttitlepage' command |
18 | @c @clear shorttitlepage-enabled | |
19 | @c @set shorttitlepage-enabled | |
20 | ||
21 | @c ISPELL CHECK: done, 14 Jan 1993 --bob | |
22 | ||
23 | @c Check COPYRIGHT dates. should be updated in the titlepage, ifinfo | |
24 | @c titlepage; should NOT be changed in the GPL. --mew | |
25 | ||
26 | @iftex | |
27 | @syncodeindex fn cp | |
28 | @syncodeindex vr cp | |
29 | @syncodeindex tp cp | |
30 | @end iftex | |
31 | @ifinfo | |
32 | @synindex fn cp | |
33 | @synindex vr cp | |
34 | @synindex tp cp | |
35 | @end ifinfo | |
36 | @comment %**end of header | |
37 | ||
bd773d73 JT |
38 | @ifinfo |
39 | @format | |
40 | START-INFO-DIR-ENTRY | |
41 | * bison: (bison). GNU Project parser generator (yacc replacement). | |
42 | END-INFO-DIR-ENTRY | |
43 | @end format | |
44 | @end ifinfo | |
45 | ||
bfa74976 RS |
46 | @ifinfo |
47 | This file documents the Bison parser generator. | |
48 | ||
79282c6c AD |
49 | Copyright (C) 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998, 1999, |
50 | 2000, 2001 | |
13863333 | 51 | Free Software Foundation, Inc. |
bfa74976 RS |
52 | |
53 | Permission is granted to make and distribute verbatim copies of | |
54 | this manual provided the copyright notice and this permission notice | |
55 | are preserved on all copies. | |
56 | ||
57 | @ignore | |
58 | Permission is granted to process this file through Tex and print the | |
59 | results, provided the printed document carries copying permission | |
60 | notice identical to this one except for the removal of this paragraph | |
61 | (this paragraph not being relevant to the printed manual). | |
62 | ||
63 | @end ignore | |
64 | Permission is granted to copy and distribute modified versions of this | |
65 | manual under the conditions for verbatim copying, provided also that the | |
66 | sections entitled ``GNU General Public License'' and ``Conditions for | |
67 | Using Bison'' are included exactly as in the original, and provided that | |
68 | the entire resulting derived work is distributed under the terms of a | |
69 | permission notice identical to this one. | |
70 | ||
71 | Permission is granted to copy and distribute translations of this manual | |
72 | into another language, under the above conditions for modified versions, | |
73 | except that the sections entitled ``GNU General Public License'', | |
74 | ``Conditions for Using Bison'' and this permission notice may be | |
75 | included in translations approved by the Free Software Foundation | |
76 | instead of in the original English. | |
77 | @end ifinfo | |
78 | ||
79 | @ifset shorttitlepage-enabled | |
80 | @shorttitlepage Bison | |
81 | @end ifset | |
82 | @titlepage | |
83 | @title Bison | |
84 | @subtitle The YACC-compatible Parser Generator | |
df1af54c | 85 | @subtitle @value{UPDATED}, Bison Version @value{VERSION} |
bfa74976 RS |
86 | |
87 | @author by Charles Donnelly and Richard Stallman | |
88 | ||
89 | @page | |
90 | @vskip 0pt plus 1filll | |
13863333 | 91 | Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998, |
79282c6c | 92 | 1999, 2000, 2001 |
13863333 | 93 | Free Software Foundation, Inc. |
bfa74976 RS |
94 | |
95 | @sp 2 | |
96 | Published by the Free Software Foundation @* | |
931c7513 RS |
97 | 59 Temple Place, Suite 330 @* |
98 | Boston, MA 02111-1307 USA @* | |
9ecbd125 JT |
99 | Printed copies are available from the Free Software Foundation.@* |
100 | ISBN 1-882114-44-2 | |
bfa74976 RS |
101 | |
102 | Permission is granted to make and distribute verbatim copies of | |
103 | this manual provided the copyright notice and this permission notice | |
104 | are preserved on all copies. | |
105 | ||
106 | @ignore | |
107 | Permission is granted to process this file through TeX and print the | |
108 | results, provided the printed document carries copying permission | |
109 | notice identical to this one except for the removal of this paragraph | |
110 | (this paragraph not being relevant to the printed manual). | |
111 | ||
112 | @end ignore | |
113 | Permission is granted to copy and distribute modified versions of this | |
114 | manual under the conditions for verbatim copying, provided also that the | |
115 | sections entitled ``GNU General Public License'' and ``Conditions for | |
116 | Using Bison'' are included exactly as in the original, and provided that | |
117 | the entire resulting derived work is distributed under the terms of a | |
118 | permission notice identical to this one. | |
119 | ||
120 | Permission is granted to copy and distribute translations of this manual | |
121 | into another language, under the above conditions for modified versions, | |
122 | except that the sections entitled ``GNU General Public License'', | |
123 | ``Conditions for Using Bison'' and this permission notice may be | |
124 | included in translations approved by the Free Software Foundation | |
125 | instead of in the original English. | |
126 | @sp 2 | |
127 | Cover art by Etienne Suvasa. | |
128 | @end titlepage | |
d5796688 JT |
129 | |
130 | @contents | |
bfa74976 RS |
131 | |
132 | @node Top, Introduction, (dir), (dir) | |
133 | ||
134 | @ifinfo | |
df1af54c | 135 | This manual documents version @value{VERSION} of Bison. |
bfa74976 RS |
136 | @end ifinfo |
137 | ||
138 | @menu | |
13863333 AD |
139 | * Introduction:: |
140 | * Conditions:: | |
bfa74976 RS |
141 | * Copying:: The GNU General Public License says |
142 | how you can copy and share Bison | |
143 | ||
144 | Tutorial sections: | |
145 | * Concepts:: Basic concepts for understanding Bison. | |
146 | * Examples:: Three simple explained examples of using Bison. | |
147 | ||
148 | Reference sections: | |
149 | * Grammar File:: Writing Bison declarations and rules. | |
150 | * Interface:: C-language interface to the parser function @code{yyparse}. | |
151 | * Algorithm:: How the Bison parser works at run-time. | |
152 | * Error Recovery:: Writing rules for error recovery. | |
153 | * Context Dependency:: What to do if your language syntax is too | |
154 | messy for Bison to handle straightforwardly. | |
155 | * Debugging:: Debugging Bison parsers that parse wrong. | |
156 | * Invocation:: How to run Bison (to produce the parser source file). | |
157 | * Table of Symbols:: All the keywords of the Bison language are explained. | |
158 | * Glossary:: Basic concepts are explained. | |
f2b5126e | 159 | * Copying This Manual:: License for copying this manual. |
bfa74976 RS |
160 | * Index:: Cross-references to the text. |
161 | ||
162 | --- The Detailed Node Listing --- | |
163 | ||
164 | The Concepts of Bison | |
165 | ||
166 | * Language and Grammar:: Languages and context-free grammars, | |
167 | as mathematical ideas. | |
168 | * Grammar in Bison:: How we represent grammars for Bison's sake. | |
169 | * Semantic Values:: Each token or syntactic grouping can have | |
170 | a semantic value (the value of an integer, | |
171 | the name of an identifier, etc.). | |
172 | * Semantic Actions:: Each rule can have an action containing C code. | |
173 | * Bison Parser:: What are Bison's input and output, | |
174 | how is the output used? | |
175 | * Stages:: Stages in writing and running Bison grammars. | |
176 | * Grammar Layout:: Overall structure of a Bison grammar file. | |
177 | ||
178 | Examples | |
179 | ||
180 | * RPN Calc:: Reverse polish notation calculator; | |
181 | a first example with no operator precedence. | |
182 | * Infix Calc:: Infix (algebraic) notation calculator. | |
183 | Operator precedence is introduced. | |
184 | * Simple Error Recovery:: Continuing after syntax errors. | |
185 | * Multi-function Calc:: Calculator with memory and trig functions. | |
186 | It uses multiple data-types for semantic values. | |
187 | * Exercises:: Ideas for improving the multi-function calculator. | |
188 | ||
189 | Reverse Polish Notation Calculator | |
190 | ||
75f5aaea | 191 | * Decls: Rpcalc Decls. Prologue (declarations) for rpcalc. |
bfa74976 RS |
192 | * Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation. |
193 | * Lexer: Rpcalc Lexer. The lexical analyzer. | |
194 | * Main: Rpcalc Main. The controlling function. | |
195 | * Error: Rpcalc Error. The error reporting function. | |
196 | * Gen: Rpcalc Gen. Running Bison on the grammar file. | |
197 | * Comp: Rpcalc Compile. Run the C compiler on the output code. | |
198 | ||
199 | Grammar Rules for @code{rpcalc} | |
200 | ||
13863333 AD |
201 | * Rpcalc Input:: |
202 | * Rpcalc Line:: | |
203 | * Rpcalc Expr:: | |
bfa74976 RS |
204 | |
205 | Multi-Function Calculator: @code{mfcalc} | |
206 | ||
207 | * Decl: Mfcalc Decl. Bison declarations for multi-function calculator. | |
208 | * Rules: Mfcalc Rules. Grammar rules for the calculator. | |
209 | * Symtab: Mfcalc Symtab. Symbol table management subroutines. | |
210 | ||
211 | Bison Grammar Files | |
212 | ||
213 | * Grammar Outline:: Overall layout of the grammar file. | |
214 | * Symbols:: Terminal and nonterminal symbols. | |
215 | * Rules:: How to write grammar rules. | |
216 | * Recursion:: Writing recursive rules. | |
217 | * Semantics:: Semantic values and actions. | |
218 | * Declarations:: All kinds of Bison declarations are described here. | |
219 | * Multiple Parsers:: Putting more than one Bison parser in one program. | |
220 | ||
221 | Outline of a Bison Grammar | |
222 | ||
75f5aaea | 223 | * Prologue:: Syntax and usage of the prologue (declarations section). |
bfa74976 RS |
224 | * Bison Declarations:: Syntax and usage of the Bison declarations section. |
225 | * Grammar Rules:: Syntax and usage of the grammar rules section. | |
75f5aaea | 226 | * Epilogue:: Syntax and usage of the epilogue (additional code section). |
bfa74976 RS |
227 | |
228 | Defining Language Semantics | |
229 | ||
230 | * Value Type:: Specifying one data type for all semantic values. | |
231 | * Multiple Types:: Specifying several alternative data types. | |
232 | * Actions:: An action is the semantic definition of a grammar rule. | |
233 | * Action Types:: Specifying data types for actions to operate on. | |
234 | * Mid-Rule Actions:: Most actions go at the end of a rule. | |
235 | This says when, why and how to use the exceptional | |
236 | action in the middle of a rule. | |
237 | ||
238 | Bison Declarations | |
239 | ||
240 | * Token Decl:: Declaring terminal symbols. | |
241 | * Precedence Decl:: Declaring terminals with precedence and associativity. | |
242 | * Union Decl:: Declaring the set of all semantic value types. | |
243 | * Type Decl:: Declaring the choice of type for a nonterminal symbol. | |
244 | * Expect Decl:: Suppressing warnings about shift/reduce conflicts. | |
245 | * Start Decl:: Specifying the start symbol. | |
246 | * Pure Decl:: Requesting a reentrant parser. | |
247 | * Decl Summary:: Table of all Bison declarations. | |
248 | ||
249 | Parser C-Language Interface | |
250 | ||
251 | * Parser Function:: How to call @code{yyparse} and what it returns. | |
13863333 | 252 | * Lexical:: You must supply a function @code{yylex} |
bfa74976 RS |
253 | which reads tokens. |
254 | * Error Reporting:: You must supply a function @code{yyerror}. | |
255 | * Action Features:: Special features for use in actions. | |
256 | ||
257 | The Lexical Analyzer Function @code{yylex} | |
258 | ||
259 | * Calling Convention:: How @code{yyparse} calls @code{yylex}. | |
260 | * Token Values:: How @code{yylex} must return the semantic value | |
261 | of the token it has read. | |
262 | * Token Positions:: How @code{yylex} must return the text position | |
263 | (line number, etc.) of the token, if the | |
264 | actions want that. | |
265 | * Pure Calling:: How the calling convention differs | |
266 | in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). | |
267 | ||
13863333 | 268 | The Bison Parser Algorithm |
bfa74976 RS |
269 | |
270 | * Look-Ahead:: Parser looks one token ahead when deciding what to do. | |
271 | * Shift/Reduce:: Conflicts: when either shifting or reduction is valid. | |
272 | * Precedence:: Operator precedence works by resolving conflicts. | |
273 | * Contextual Precedence:: When an operator's precedence depends on context. | |
274 | * Parser States:: The parser is a finite-state-machine with stack. | |
275 | * Reduce/Reduce:: When two rules are applicable in the same situation. | |
276 | * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified. | |
277 | * Stack Overflow:: What happens when stack gets full. How to avoid it. | |
278 | ||
279 | Operator Precedence | |
280 | ||
281 | * Why Precedence:: An example showing why precedence is needed. | |
282 | * Using Precedence:: How to specify precedence in Bison grammars. | |
283 | * Precedence Examples:: How these features are used in the previous example. | |
284 | * How Precedence:: How they work. | |
285 | ||
286 | Handling Context Dependencies | |
287 | ||
288 | * Semantic Tokens:: Token parsing can depend on the semantic context. | |
289 | * Lexical Tie-ins:: Token parsing can depend on the syntactic context. | |
290 | * Tie-in Recovery:: Lexical tie-ins have implications for how | |
291 | error recovery rules must be written. | |
292 | ||
293 | Invoking Bison | |
294 | ||
13863333 | 295 | * Bison Options:: All the options described in detail, |
bfa74976 RS |
296 | in alphabetical order by short options. |
297 | * Option Cross Key:: Alphabetical list of long options. | |
298 | * VMS Invocation:: Bison command syntax on VMS. | |
f2b5126e PB |
299 | |
300 | Copying This Manual | |
301 | ||
302 | * GNU Free Documentation License:: License for copying this manual. | |
303 | ||
bfa74976 RS |
304 | @end menu |
305 | ||
306 | @node Introduction, Conditions, Top, Top | |
307 | @unnumbered Introduction | |
308 | @cindex introduction | |
309 | ||
310 | @dfn{Bison} is a general-purpose parser generator that converts a | |
311 | grammar description for an LALR(1) context-free grammar into a C | |
312 | program to parse that grammar. Once you are proficient with Bison, | |
313 | you may use it to develop a wide range of language parsers, from those | |
314 | used in simple desk calculators to complex programming languages. | |
315 | ||
316 | Bison is upward compatible with Yacc: all properly-written Yacc grammars | |
317 | ought to work with Bison with no change. Anyone familiar with Yacc | |
318 | should be able to use Bison with little trouble. You need to be fluent in | |
319 | C programming in order to use Bison or to understand this manual. | |
320 | ||
321 | We begin with tutorial chapters that explain the basic concepts of using | |
322 | Bison and show three explained examples, each building on the last. If you | |
323 | don't know Bison or Yacc, start by reading these chapters. Reference | |
324 | chapters follow which describe specific aspects of Bison in detail. | |
325 | ||
931c7513 RS |
326 | Bison was written primarily by Robert Corbett; Richard Stallman made it |
327 | Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added | |
14ded682 | 328 | multi-character string literals and other features. |
931c7513 | 329 | |
df1af54c | 330 | This edition corresponds to version @value{VERSION} of Bison. |
bfa74976 RS |
331 | |
332 | @node Conditions, Copying, Introduction, Top | |
333 | @unnumbered Conditions for Using Bison | |
334 | ||
a31239f1 | 335 | As of Bison version 1.24, we have changed the distribution terms for |
9ecbd125 | 336 | @code{yyparse} to permit using Bison's output in nonfree programs. |
a31239f1 RS |
337 | Formerly, Bison parsers could be used only in programs that were free |
338 | software. | |
339 | ||
340 | The other GNU programming tools, such as the GNU C compiler, have never | |
9ecbd125 | 341 | had such a requirement. They could always be used for nonfree |
a31239f1 RS |
342 | software. The reason Bison was different was not due to a special |
343 | policy decision; it resulted from applying the usual General Public | |
344 | License to all of the Bison source code. | |
345 | ||
346 | The output of the Bison utility---the Bison parser file---contains a | |
347 | verbatim copy of a sizable piece of Bison, which is the code for the | |
348 | @code{yyparse} function. (The actions from your grammar are inserted | |
349 | into this function at one point, but the rest of the function is not | |
350 | changed.) When we applied the GPL terms to the code for @code{yyparse}, | |
351 | the effect was to restrict the use of Bison output to free software. | |
352 | ||
353 | We didn't change the terms because of sympathy for people who want to | |
354 | make software proprietary. @strong{Software should be free.} But we | |
355 | concluded that limiting Bison's use to free software was doing little to | |
356 | encourage people to make other software free. So we decided to make the | |
357 | practical conditions for using Bison match the practical conditions for | |
358 | using the other GNU tools. | |
bfa74976 | 359 | |
c67a198d | 360 | @include gpl.texi |
bfa74976 RS |
361 | |
362 | @node Concepts, Examples, Copying, Top | |
363 | @chapter The Concepts of Bison | |
364 | ||
365 | This chapter introduces many of the basic concepts without which the | |
366 | details of Bison will not make sense. If you do not already know how to | |
367 | use Bison or Yacc, we suggest you start by reading this chapter carefully. | |
368 | ||
369 | @menu | |
370 | * Language and Grammar:: Languages and context-free grammars, | |
371 | as mathematical ideas. | |
372 | * Grammar in Bison:: How we represent grammars for Bison's sake. | |
373 | * Semantic Values:: Each token or syntactic grouping can have | |
374 | a semantic value (the value of an integer, | |
375 | the name of an identifier, etc.). | |
376 | * Semantic Actions:: Each rule can have an action containing C code. | |
847bf1f5 | 377 | * Locations Overview:: Tracking Locations. |
bfa74976 RS |
378 | * Bison Parser:: What are Bison's input and output, |
379 | how is the output used? | |
380 | * Stages:: Stages in writing and running Bison grammars. | |
381 | * Grammar Layout:: Overall structure of a Bison grammar file. | |
382 | @end menu | |
383 | ||
384 | @node Language and Grammar, Grammar in Bison, , Concepts | |
385 | @section Languages and Context-Free Grammars | |
386 | ||
bfa74976 RS |
387 | @cindex context-free grammar |
388 | @cindex grammar, context-free | |
389 | In order for Bison to parse a language, it must be described by a | |
390 | @dfn{context-free grammar}. This means that you specify one or more | |
391 | @dfn{syntactic groupings} and give rules for constructing them from their | |
392 | parts. For example, in the C language, one kind of grouping is called an | |
393 | `expression'. One rule for making an expression might be, ``An expression | |
394 | can be made of a minus sign and another expression''. Another would be, | |
395 | ``An expression can be an integer''. As you can see, rules are often | |
396 | recursive, but there must be at least one rule which leads out of the | |
397 | recursion. | |
398 | ||
399 | @cindex BNF | |
400 | @cindex Backus-Naur form | |
401 | The most common formal system for presenting such rules for humans to read | |
402 | is @dfn{Backus-Naur Form} or ``BNF'', which was developed in order to | |
403 | specify the language Algol 60. Any grammar expressed in BNF is a | |
404 | context-free grammar. The input to Bison is essentially machine-readable | |
405 | BNF. | |
406 | ||
407 | Not all context-free languages can be handled by Bison, only those | |
408 | that are LALR(1). In brief, this means that it must be possible to | |
409 | tell how to parse any portion of an input string with just a single | |
410 | token of look-ahead. Strictly speaking, that is a description of an | |
411 | LR(1) grammar, and LALR(1) involves additional restrictions that are | |
412 | hard to explain simply; but it is rare in actual practice to find an | |
413 | LR(1) grammar that fails to be LALR(1). @xref{Mystery Conflicts, , | |
414 | Mysterious Reduce/Reduce Conflicts}, for more information on this. | |
415 | ||
416 | @cindex symbols (abstract) | |
417 | @cindex token | |
418 | @cindex syntactic grouping | |
419 | @cindex grouping, syntactic | |
420 | In the formal grammatical rules for a language, each kind of syntactic unit | |
421 | or grouping is named by a @dfn{symbol}. Those which are built by grouping | |
422 | smaller constructs according to grammatical rules are called | |
423 | @dfn{nonterminal symbols}; those which can't be subdivided are called | |
424 | @dfn{terminal symbols} or @dfn{token types}. We call a piece of input | |
425 | corresponding to a single terminal symbol a @dfn{token}, and a piece | |
426 | corresponding to a single nonterminal symbol a @dfn{grouping}.@refill | |
427 | ||
428 | We can use the C language as an example of what symbols, terminal and | |
429 | nonterminal, mean. The tokens of C are identifiers, constants (numeric and | |
430 | string), and the various keywords, arithmetic operators and punctuation | |
431 | marks. So the terminal symbols of a grammar for C include `identifier', | |
432 | `number', `string', plus one symbol for each keyword, operator or | |
433 | punctuation mark: `if', `return', `const', `static', `int', `char', | |
434 | `plus-sign', `open-brace', `close-brace', `comma' and many more. (These | |
435 | tokens can be subdivided into characters, but that is a matter of | |
436 | lexicography, not grammar.) | |
437 | ||
438 | Here is a simple C function subdivided into tokens: | |
439 | ||
440 | @example | |
441 | int /* @r{keyword `int'} */ | |
442 | square (x) /* @r{identifier, open-paren,} */ | |
443 | /* @r{identifier, close-paren} */ | |
444 | int x; /* @r{keyword `int', identifier, semicolon} */ | |
445 | @{ /* @r{open-brace} */ | |
446 | return x * x; /* @r{keyword `return', identifier,} */ | |
447 | /* @r{asterisk, identifier, semicolon} */ | |
448 | @} /* @r{close-brace} */ | |
449 | @end example | |
450 | ||
451 | The syntactic groupings of C include the expression, the statement, the | |
452 | declaration, and the function definition. These are represented in the | |
453 | grammar of C by nonterminal symbols `expression', `statement', | |
454 | `declaration' and `function definition'. The full grammar uses dozens of | |
455 | additional language constructs, each with its own nonterminal symbol, in | |
456 | order to express the meanings of these four. The example above is a | |
457 | function definition; it contains one declaration, and one statement. In | |
458 | the statement, each @samp{x} is an expression and so is @samp{x * x}. | |
459 | ||
460 | Each nonterminal symbol must have grammatical rules showing how it is made | |
461 | out of simpler constructs. For example, one kind of C statement is the | |
462 | @code{return} statement; this would be described with a grammar rule which | |
463 | reads informally as follows: | |
464 | ||
465 | @quotation | |
466 | A `statement' can be made of a `return' keyword, an `expression' and a | |
467 | `semicolon'. | |
468 | @end quotation | |
469 | ||
470 | @noindent | |
471 | There would be many other rules for `statement', one for each kind of | |
472 | statement in C. | |
473 | ||
474 | @cindex start symbol | |
475 | One nonterminal symbol must be distinguished as the special one which | |
476 | defines a complete utterance in the language. It is called the @dfn{start | |
477 | symbol}. In a compiler, this means a complete input program. In the C | |
478 | language, the nonterminal symbol `sequence of definitions and declarations' | |
479 | plays this role. | |
480 | ||
481 | For example, @samp{1 + 2} is a valid C expression---a valid part of a C | |
482 | program---but it is not valid as an @emph{entire} C program. In the | |
483 | context-free grammar of C, this follows from the fact that `expression' is | |
484 | not the start symbol. | |
485 | ||
486 | The Bison parser reads a sequence of tokens as its input, and groups the | |
487 | tokens using the grammar rules. If the input is valid, the end result is | |
488 | that the entire token sequence reduces to a single grouping whose symbol is | |
489 | the grammar's start symbol. If we use a grammar for C, the entire input | |
490 | must be a `sequence of definitions and declarations'. If not, the parser | |
491 | reports a syntax error. | |
492 | ||
493 | @node Grammar in Bison, Semantic Values, Language and Grammar, Concepts | |
494 | @section From Formal Rules to Bison Input | |
495 | @cindex Bison grammar | |
496 | @cindex grammar, Bison | |
497 | @cindex formal grammar | |
498 | ||
499 | A formal grammar is a mathematical construct. To define the language | |
500 | for Bison, you must write a file expressing the grammar in Bison syntax: | |
501 | a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}. | |
502 | ||
503 | A nonterminal symbol in the formal grammar is represented in Bison input | |
504 | as an identifier, like an identifier in C. By convention, it should be | |
505 | in lower case, such as @code{expr}, @code{stmt} or @code{declaration}. | |
506 | ||
507 | The Bison representation for a terminal symbol is also called a @dfn{token | |
508 | type}. Token types as well can be represented as C-like identifiers. By | |
509 | convention, these identifiers should be upper case to distinguish them from | |
510 | nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or | |
511 | @code{RETURN}. A terminal symbol that stands for a particular keyword in | |
512 | the language should be named after that keyword converted to upper case. | |
513 | The terminal symbol @code{error} is reserved for error recovery. | |
931c7513 | 514 | @xref{Symbols}. |
bfa74976 RS |
515 | |
516 | A terminal symbol can also be represented as a character literal, just like | |
517 | a C character constant. You should do this whenever a token is just a | |
518 | single character (parenthesis, plus-sign, etc.): use that same character in | |
519 | a literal as the terminal symbol for that token. | |
520 | ||
931c7513 RS |
521 | A third way to represent a terminal symbol is with a C string constant |
522 | containing several characters. @xref{Symbols}, for more information. | |
523 | ||
bfa74976 RS |
524 | The grammar rules also have an expression in Bison syntax. For example, |
525 | here is the Bison rule for a C @code{return} statement. The semicolon in | |
526 | quotes is a literal character token, representing part of the C syntax for | |
527 | the statement; the naked semicolon, and the colon, are Bison punctuation | |
528 | used in every rule. | |
529 | ||
530 | @example | |
531 | stmt: RETURN expr ';' | |
532 | ; | |
533 | @end example | |
534 | ||
535 | @noindent | |
536 | @xref{Rules, ,Syntax of Grammar Rules}. | |
537 | ||
538 | @node Semantic Values, Semantic Actions, Grammar in Bison, Concepts | |
539 | @section Semantic Values | |
540 | @cindex semantic value | |
541 | @cindex value, semantic | |
542 | ||
543 | A formal grammar selects tokens only by their classifications: for example, | |
544 | if a rule mentions the terminal symbol `integer constant', it means that | |
545 | @emph{any} integer constant is grammatically valid in that position. The | |
546 | precise value of the constant is irrelevant to how to parse the input: if | |
547 | @samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally | |
548 | grammatical.@refill | |
549 | ||
550 | But the precise value is very important for what the input means once it is | |
551 | parsed. A compiler is useless if it fails to distinguish between 4, 1 and | |
552 | 3989 as constants in the program! Therefore, each token in a Bison grammar | |
553 | has both a token type and a @dfn{semantic value}. @xref{Semantics, ,Defining Language Semantics}, | |
554 | for details. | |
555 | ||
556 | The token type is a terminal symbol defined in the grammar, such as | |
557 | @code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything | |
558 | you need to know to decide where the token may validly appear and how to | |
559 | group it with other tokens. The grammar rules know nothing about tokens | |
560 | except their types.@refill | |
561 | ||
562 | The semantic value has all the rest of the information about the | |
563 | meaning of the token, such as the value of an integer, or the name of an | |
564 | identifier. (A token such as @code{','} which is just punctuation doesn't | |
565 | need to have any semantic value.) | |
566 | ||
567 | For example, an input token might be classified as token type | |
568 | @code{INTEGER} and have the semantic value 4. Another input token might | |
569 | have the same token type @code{INTEGER} but value 3989. When a grammar | |
570 | rule says that @code{INTEGER} is allowed, either of these tokens is | |
571 | acceptable because each is an @code{INTEGER}. When the parser accepts the | |
572 | token, it keeps track of the token's semantic value. | |
573 | ||
574 | Each grouping can also have a semantic value as well as its nonterminal | |
575 | symbol. For example, in a calculator, an expression typically has a | |
576 | semantic value that is a number. In a compiler for a programming | |
577 | language, an expression typically has a semantic value that is a tree | |
578 | structure describing the meaning of the expression. | |
579 | ||
847bf1f5 | 580 | @node Semantic Actions, Locations Overview, Semantic Values, Concepts |
bfa74976 RS |
581 | @section Semantic Actions |
582 | @cindex semantic actions | |
583 | @cindex actions, semantic | |
584 | ||
585 | In order to be useful, a program must do more than parse input; it must | |
586 | also produce some output based on the input. In a Bison grammar, a grammar | |
587 | rule can have an @dfn{action} made up of C statements. Each time the | |
588 | parser recognizes a match for that rule, the action is executed. | |
589 | @xref{Actions}. | |
13863333 | 590 | |
bfa74976 RS |
591 | Most of the time, the purpose of an action is to compute the semantic value |
592 | of the whole construct from the semantic values of its parts. For example, | |
593 | suppose we have a rule which says an expression can be the sum of two | |
594 | expressions. When the parser recognizes such a sum, each of the | |
595 | subexpressions has a semantic value which describes how it was built up. | |
596 | The action for this rule should create a similar sort of value for the | |
597 | newly recognized larger expression. | |
598 | ||
599 | For example, here is a rule that says an expression can be the sum of | |
600 | two subexpressions: | |
601 | ||
602 | @example | |
603 | expr: expr '+' expr @{ $$ = $1 + $3; @} | |
604 | ; | |
605 | @end example | |
606 | ||
607 | @noindent | |
608 | The action says how to produce the semantic value of the sum expression | |
609 | from the values of the two subexpressions. | |
610 | ||
847bf1f5 AD |
611 | @node Locations Overview, Bison Parser, Semantic Actions, Concepts |
612 | @section Locations | |
613 | @cindex location | |
614 | @cindex textual position | |
615 | @cindex position, textual | |
616 | ||
617 | Many applications, like interpreters or compilers, have to produce verbose | |
618 | and useful error messages. To achieve this, one must be able to keep track of | |
619 | the @dfn{textual position}, or @dfn{location}, of each syntactic construct. | |
620 | Bison provides a mechanism for handling these locations. | |
621 | ||
622 | Each token has a semantic value. In a similar fashion, each token has an | |
623 | associated location, but the type of locations is the same for all tokens and | |
624 | groupings. Moreover, the output parser is equipped with a default data | |
625 | structure for storing locations (@pxref{Locations}, for more details). | |
626 | ||
627 | Like semantic values, locations can be reached in actions using a dedicated | |
628 | set of constructs. In the example above, the location of the whole grouping | |
629 | is @code{@@$}, while the locations of the subexpressions are @code{@@1} and | |
630 | @code{@@3}. | |
631 | ||
632 | When a rule is matched, a default action is used to compute the semantic value | |
633 | of its left hand side (@pxref{Actions}). In the same way, another default | |
634 | action is used for locations. However, the action for locations is general | |
635 | enough for most cases, meaning there is usually no need to describe for each | |
636 | rule how @code{@@$} should be formed. When building a new location for a given | |
637 | grouping, the default behavior of the output parser is to take the beginning | |
638 | of the first symbol, and the end of the last symbol. | |
639 | ||
640 | @node Bison Parser, Stages, Locations Overview, Concepts | |
bfa74976 RS |
641 | @section Bison Output: the Parser File |
642 | @cindex Bison parser | |
643 | @cindex Bison utility | |
644 | @cindex lexical analyzer, purpose | |
645 | @cindex parser | |
646 | ||
647 | When you run Bison, you give it a Bison grammar file as input. The output | |
648 | is a C source file that parses the language described by the grammar. | |
649 | This file is called a @dfn{Bison parser}. Keep in mind that the Bison | |
650 | utility and the Bison parser are two distinct programs: the Bison utility | |
651 | is a program whose output is the Bison parser that becomes part of your | |
652 | program. | |
653 | ||
654 | The job of the Bison parser is to group tokens into groupings according to | |
655 | the grammar rules---for example, to build identifiers and operators into | |
656 | expressions. As it does this, it runs the actions for the grammar rules it | |
657 | uses. | |
658 | ||
659 | The tokens come from a function called the @dfn{lexical analyzer} that you | |
660 | must supply in some fashion (such as by writing it in C). The Bison parser | |
661 | calls the lexical analyzer each time it wants a new token. It doesn't know | |
662 | what is ``inside'' the tokens (though their semantic values may reflect | |
663 | this). Typically the lexical analyzer makes the tokens by parsing | |
664 | characters of text, but Bison does not depend on this. @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}. | |
665 | ||
666 | The Bison parser file is C code which defines a function named | |
667 | @code{yyparse} which implements that grammar. This function does not make | |
668 | a complete C program: you must supply some additional functions. One is | |
669 | the lexical analyzer. Another is an error-reporting function which the | |
670 | parser calls to report an error. In addition, a complete C program must | |
671 | start with a function called @code{main}; you have to provide this, and | |
672 | arrange for it to call @code{yyparse} or the parser will never run. | |
673 | @xref{Interface, ,Parser C-Language Interface}. | |
674 | ||
675 | Aside from the token type names and the symbols in the actions you | |
676 | write, all variable and function names used in the Bison parser file | |
677 | begin with @samp{yy} or @samp{YY}. This includes interface functions | |
678 | such as the lexical analyzer function @code{yylex}, the error reporting | |
679 | function @code{yyerror} and the parser function @code{yyparse} itself. | |
680 | This also includes numerous identifiers used for internal purposes. | |
681 | Therefore, you should avoid using C identifiers starting with @samp{yy} | |
682 | or @samp{YY} in the Bison grammar file except for the ones defined in | |
683 | this manual. | |
684 | ||
685 | @node Stages, Grammar Layout, Bison Parser, Concepts | |
686 | @section Stages in Using Bison | |
687 | @cindex stages in using Bison | |
688 | @cindex using Bison | |
689 | ||
690 | The actual language-design process using Bison, from grammar specification | |
691 | to a working compiler or interpreter, has these parts: | |
692 | ||
693 | @enumerate | |
694 | @item | |
695 | Formally specify the grammar in a form recognized by Bison | |
696 | (@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule in the language, | |
697 | describe the action that is to be taken when an instance of that rule | |
698 | is recognized. The action is described by a sequence of C statements. | |
699 | ||
700 | @item | |
701 | Write a lexical analyzer to process input and pass tokens to the | |
702 | parser. The lexical analyzer may be written by hand in C | |
703 | (@pxref{Lexical, ,The Lexical Analyzer Function @code{yylex}}). It could also be produced using Lex, but the use | |
704 | of Lex is not discussed in this manual. | |
705 | ||
706 | @item | |
707 | Write a controlling function that calls the Bison-produced parser. | |
708 | ||
709 | @item | |
710 | Write error-reporting routines. | |
711 | @end enumerate | |
712 | ||
713 | To turn this source code as written into a runnable program, you | |
714 | must follow these steps: | |
715 | ||
716 | @enumerate | |
717 | @item | |
718 | Run Bison on the grammar to produce the parser. | |
719 | ||
720 | @item | |
721 | Compile the code output by Bison, as well as any other source files. | |
722 | ||
723 | @item | |
724 | Link the object files to produce the finished product. | |
725 | @end enumerate | |
726 | ||
727 | @node Grammar Layout, , Stages, Concepts | |
728 | @section The Overall Layout of a Bison Grammar | |
729 | @cindex grammar file | |
730 | @cindex file format | |
731 | @cindex format of grammar file | |
732 | @cindex layout of Bison grammar | |
733 | ||
734 | The input file for the Bison utility is a @dfn{Bison grammar file}. The | |
735 | general form of a Bison grammar file is as follows: | |
736 | ||
737 | @example | |
738 | %@{ | |
75f5aaea | 739 | @var{Prologue (declarations)} |
bfa74976 RS |
740 | %@} |
741 | ||
742 | @var{Bison declarations} | |
743 | ||
744 | %% | |
745 | @var{Grammar rules} | |
746 | %% | |
75f5aaea | 747 | @var{Epilogue (additional code)} |
bfa74976 RS |
748 | @end example |
749 | ||
750 | @noindent | |
751 | The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears | |
752 | in every Bison grammar file to separate the sections. | |
753 | ||
75f5aaea MA |
754 | The prologue may define types and variables used in the actions. You can |
755 | also use preprocessor commands to define macros used there, and use | |
bfa74976 RS |
756 | @code{#include} to include header files that do any of these things. |
757 | ||
758 | The Bison declarations declare the names of the terminal and nonterminal | |
759 | symbols, and may also describe operator precedence and the data types of | |
760 | semantic values of various symbols. | |
761 | ||
762 | The grammar rules define how to construct each nonterminal symbol from its | |
763 | parts. | |
764 | ||
75f5aaea MA |
765 | The epilogue can contain any code you want to use. Often the definition of |
766 | the lexical analyzer @code{yylex} goes here, plus subroutines called by the | |
767 | actions in the grammar rules. In a simple program, all the rest of the | |
768 | program can go here. | |
bfa74976 RS |
769 | |
770 | @node Examples, Grammar File, Concepts, Top | |
771 | @chapter Examples | |
772 | @cindex simple examples | |
773 | @cindex examples, simple | |
774 | ||
775 | Now we show and explain three sample programs written using Bison: a | |
776 | reverse polish notation calculator, an algebraic (infix) notation | |
777 | calculator, and a multi-function calculator. All three have been tested | |
778 | under BSD Unix 4.3; each produces a usable, though limited, interactive | |
779 | desk-top calculator. | |
780 | ||
781 | These examples are simple, but Bison grammars for real programming | |
782 | languages are written the same way. | |
783 | @ifinfo | |
784 | You can copy these examples out of the Info file and into a source file | |
785 | to try them. | |
786 | @end ifinfo | |
787 | ||
788 | @menu | |
789 | * RPN Calc:: Reverse polish notation calculator; | |
790 | a first example with no operator precedence. | |
791 | * Infix Calc:: Infix (algebraic) notation calculator. | |
792 | Operator precedence is introduced. | |
793 | * Simple Error Recovery:: Continuing after syntax errors. | |
794 | * Multi-function Calc:: Calculator with memory and trig functions. | |
795 | It uses multiple data-types for semantic values. | |
796 | * Exercises:: Ideas for improving the multi-function calculator. | |
797 | @end menu | |
798 | ||
799 | @node RPN Calc, Infix Calc, , Examples | |
800 | @section Reverse Polish Notation Calculator | |
801 | @cindex reverse polish notation | |
802 | @cindex polish notation calculator | |
803 | @cindex @code{rpcalc} | |
804 | @cindex calculator, simple | |
805 | ||
806 | The first example is that of a simple double-precision @dfn{reverse polish | |
807 | notation} calculator (a calculator using postfix operators). This example | |
808 | provides a good starting point, since operator precedence is not an issue. | |
809 | The second example will illustrate how operator precedence is handled. | |
810 | ||
811 | The source code for this calculator is named @file{rpcalc.y}. The | |
812 | @samp{.y} extension is a convention used for Bison input files. | |
813 | ||
814 | @menu | |
75f5aaea | 815 | * Decls: Rpcalc Decls. Prologue (declarations) for rpcalc. |
bfa74976 RS |
816 | * Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation. |
817 | * Lexer: Rpcalc Lexer. The lexical analyzer. | |
818 | * Main: Rpcalc Main. The controlling function. | |
819 | * Error: Rpcalc Error. The error reporting function. | |
820 | * Gen: Rpcalc Gen. Running Bison on the grammar file. | |
821 | * Comp: Rpcalc Compile. Run the C compiler on the output code. | |
822 | @end menu | |
823 | ||
824 | @node Rpcalc Decls, Rpcalc Rules, , RPN Calc | |
825 | @subsection Declarations for @code{rpcalc} | |
826 | ||
827 | Here are the C and Bison declarations for the reverse polish notation | |
828 | calculator. As in C, comments are placed between @samp{/*@dots{}*/}. | |
829 | ||
830 | @example | |
831 | /* Reverse polish notation calculator. */ | |
832 | ||
833 | %@{ | |
834 | #define YYSTYPE double | |
835 | #include <math.h> | |
836 | %@} | |
837 | ||
838 | %token NUM | |
839 | ||
840 | %% /* Grammar rules and actions follow */ | |
841 | @end example | |
842 | ||
75f5aaea | 843 | The declarations section (@pxref{Prologue, , The prologue}) contains two |
bfa74976 RS |
844 | preprocessor directives. |
845 | ||
846 | The @code{#define} directive defines the macro @code{YYSTYPE}, thus | |
847 | specifying the C data type for semantic values of both tokens and groupings | |
848 | (@pxref{Value Type, ,Data Types of Semantic Values}). The Bison parser will use whatever type | |
849 | @code{YYSTYPE} is defined as; if you don't define it, @code{int} is the | |
850 | default. Because we specify @code{double}, each token and each expression | |
851 | has an associated value, which is a floating point number. | |
852 | ||
853 | The @code{#include} directive is used to declare the exponentiation | |
854 | function @code{pow}. | |
855 | ||
856 | The second section, Bison declarations, provides information to Bison about | |
857 | the token types (@pxref{Bison Declarations, ,The Bison Declarations Section}). Each terminal symbol that is | |
858 | not a single-character literal must be declared here. (Single-character | |
859 | literals normally don't need to be declared.) In this example, all the | |
860 | arithmetic operators are designated by single-character literals, so the | |
861 | only terminal symbol that needs to be declared is @code{NUM}, the token | |
862 | type for numeric constants. | |
863 | ||
864 | @node Rpcalc Rules, Rpcalc Lexer, Rpcalc Decls, RPN Calc | |
865 | @subsection Grammar Rules for @code{rpcalc} | |
866 | ||
867 | Here are the grammar rules for the reverse polish notation calculator. | |
868 | ||
869 | @example | |
870 | input: /* empty */ | |
871 | | input line | |
872 | ; | |
873 | ||
874 | line: '\n' | |
875 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
876 | ; | |
877 | ||
878 | exp: NUM @{ $$ = $1; @} | |
879 | | exp exp '+' @{ $$ = $1 + $2; @} | |
880 | | exp exp '-' @{ $$ = $1 - $2; @} | |
881 | | exp exp '*' @{ $$ = $1 * $2; @} | |
882 | | exp exp '/' @{ $$ = $1 / $2; @} | |
883 | /* Exponentiation */ | |
884 | | exp exp '^' @{ $$ = pow ($1, $2); @} | |
885 | /* Unary minus */ | |
886 | | exp 'n' @{ $$ = -$1; @} | |
887 | ; | |
888 | %% | |
889 | @end example | |
890 | ||
891 | The groupings of the rpcalc ``language'' defined here are the expression | |
892 | (given the name @code{exp}), the line of input (@code{line}), and the | |
893 | complete input transcript (@code{input}). Each of these nonterminal | |
894 | symbols has several alternate rules, joined by the @samp{|} punctuator | |
895 | which is read as ``or''. The following sections explain what these rules | |
896 | mean. | |
897 | ||
898 | The semantics of the language is determined by the actions taken when a | |
899 | grouping is recognized. The actions are the C code that appears inside | |
900 | braces. @xref{Actions}. | |
901 | ||
902 | You must specify these actions in C, but Bison provides the means for | |
903 | passing semantic values between the rules. In each action, the | |
904 | pseudo-variable @code{$$} stands for the semantic value for the grouping | |
905 | that the rule is going to construct. Assigning a value to @code{$$} is the | |
906 | main job of most actions. The semantic values of the components of the | |
907 | rule are referred to as @code{$1}, @code{$2}, and so on. | |
908 | ||
909 | @menu | |
13863333 AD |
910 | * Rpcalc Input:: |
911 | * Rpcalc Line:: | |
912 | * Rpcalc Expr:: | |
bfa74976 RS |
913 | @end menu |
914 | ||
915 | @node Rpcalc Input, Rpcalc Line, , Rpcalc Rules | |
916 | @subsubsection Explanation of @code{input} | |
917 | ||
918 | Consider the definition of @code{input}: | |
919 | ||
920 | @example | |
921 | input: /* empty */ | |
922 | | input line | |
923 | ; | |
924 | @end example | |
925 | ||
926 | This definition reads as follows: ``A complete input is either an empty | |
927 | string, or a complete input followed by an input line''. Notice that | |
928 | ``complete input'' is defined in terms of itself. This definition is said | |
929 | to be @dfn{left recursive} since @code{input} appears always as the | |
930 | leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}. | |
931 | ||
932 | The first alternative is empty because there are no symbols between the | |
933 | colon and the first @samp{|}; this means that @code{input} can match an | |
934 | empty string of input (no tokens). We write the rules this way because it | |
935 | is legitimate to type @kbd{Ctrl-d} right after you start the calculator. | |
936 | It's conventional to put an empty alternative first and write the comment | |
937 | @samp{/* empty */} in it. | |
938 | ||
939 | The second alternate rule (@code{input line}) handles all nontrivial input. | |
940 | It means, ``After reading any number of lines, read one more line if | |
941 | possible.'' The left recursion makes this rule into a loop. Since the | |
942 | first alternative matches empty input, the loop can be executed zero or | |
943 | more times. | |
944 | ||
945 | The parser function @code{yyparse} continues to process input until a | |
946 | grammatical error is seen or the lexical analyzer says there are no more | |
947 | input tokens; we will arrange for the latter to happen at end of file. | |
948 | ||
949 | @node Rpcalc Line, Rpcalc Expr, Rpcalc Input, Rpcalc Rules | |
950 | @subsubsection Explanation of @code{line} | |
951 | ||
952 | Now consider the definition of @code{line}: | |
953 | ||
954 | @example | |
955 | line: '\n' | |
956 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
957 | ; | |
958 | @end example | |
959 | ||
960 | The first alternative is a token which is a newline character; this means | |
961 | that rpcalc accepts a blank line (and ignores it, since there is no | |
962 | action). The second alternative is an expression followed by a newline. | |
963 | This is the alternative that makes rpcalc useful. The semantic value of | |
964 | the @code{exp} grouping is the value of @code{$1} because the @code{exp} in | |
965 | question is the first symbol in the alternative. The action prints this | |
966 | value, which is the result of the computation the user asked for. | |
967 | ||
968 | This action is unusual because it does not assign a value to @code{$$}. As | |
969 | a consequence, the semantic value associated with the @code{line} is | |
970 | uninitialized (its value will be unpredictable). This would be a bug if | |
971 | that value were ever used, but we don't use it: once rpcalc has printed the | |
972 | value of the user's input line, that value is no longer needed. | |
973 | ||
974 | @node Rpcalc Expr, , Rpcalc Line, Rpcalc Rules | |
975 | @subsubsection Explanation of @code{expr} | |
976 | ||
977 | The @code{exp} grouping has several rules, one for each kind of expression. | |
978 | The first rule handles the simplest expressions: those that are just numbers. | |
979 | The second handles an addition-expression, which looks like two expressions | |
980 | followed by a plus-sign. The third handles subtraction, and so on. | |
981 | ||
982 | @example | |
983 | exp: NUM | |
984 | | exp exp '+' @{ $$ = $1 + $2; @} | |
985 | | exp exp '-' @{ $$ = $1 - $2; @} | |
986 | @dots{} | |
987 | ; | |
988 | @end example | |
989 | ||
990 | We have used @samp{|} to join all the rules for @code{exp}, but we could | |
991 | equally well have written them separately: | |
992 | ||
993 | @example | |
994 | exp: NUM ; | |
995 | exp: exp exp '+' @{ $$ = $1 + $2; @} ; | |
996 | exp: exp exp '-' @{ $$ = $1 - $2; @} ; | |
997 | @dots{} | |
998 | @end example | |
999 | ||
1000 | Most of the rules have actions that compute the value of the expression in | |
1001 | terms of the value of its parts. For example, in the rule for addition, | |
1002 | @code{$1} refers to the first component @code{exp} and @code{$2} refers to | |
1003 | the second one. The third component, @code{'+'}, has no meaningful | |
1004 | associated semantic value, but if it had one you could refer to it as | |
1005 | @code{$3}. When @code{yyparse} recognizes a sum expression using this | |
1006 | rule, the sum of the two subexpressions' values is produced as the value of | |
1007 | the entire expression. @xref{Actions}. | |
1008 | ||
1009 | You don't have to give an action for every rule. When a rule has no | |
1010 | action, Bison by default copies the value of @code{$1} into @code{$$}. | |
1011 | This is what happens in the first rule (the one that uses @code{NUM}). | |
1012 | ||
1013 | The formatting shown here is the recommended convention, but Bison does | |
1014 | not require it. You can add or change whitespace as much as you wish. | |
1015 | For example, this: | |
1016 | ||
1017 | @example | |
1018 | exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} | |
1019 | @end example | |
1020 | ||
1021 | @noindent | |
1022 | means the same thing as this: | |
1023 | ||
1024 | @example | |
1025 | exp: NUM | |
1026 | | exp exp '+' @{ $$ = $1 + $2; @} | |
1027 | | @dots{} | |
1028 | @end example | |
1029 | ||
1030 | @noindent | |
1031 | The latter, however, is much more readable. | |
1032 | ||
1033 | @node Rpcalc Lexer, Rpcalc Main, Rpcalc Rules, RPN Calc | |
1034 | @subsection The @code{rpcalc} Lexical Analyzer | |
1035 | @cindex writing a lexical analyzer | |
1036 | @cindex lexical analyzer, writing | |
1037 | ||
1038 | The lexical analyzer's job is low-level parsing: converting characters or | |
1039 | sequences of characters into tokens. The Bison parser gets its tokens by | |
1040 | calling the lexical analyzer. @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}. | |
1041 | ||
1042 | Only a simple lexical analyzer is needed for the RPN calculator. This | |
1043 | lexical analyzer skips blanks and tabs, then reads in numbers as | |
1044 | @code{double} and returns them as @code{NUM} tokens. Any other character | |
1045 | that isn't part of a number is a separate token. Note that the token-code | |
1046 | for such a single-character token is the character itself. | |
1047 | ||
1048 | The return value of the lexical analyzer function is a numeric code which | |
1049 | represents a token type. The same text used in Bison rules to stand for | |
1050 | this token type is also a C expression for the numeric code for the type. | |
1051 | This works in two ways. If the token type is a character literal, then its | |
1052 | numeric code is the ASCII code for that character; you can use the same | |
1053 | character literal in the lexical analyzer to express the number. If the | |
1054 | token type is an identifier, that identifier is defined by Bison as a C | |
1055 | macro whose definition is the appropriate number. In this example, | |
1056 | therefore, @code{NUM} becomes a macro for @code{yylex} to use. | |
1057 | ||
1058 | The semantic value of the token (if it has one) is stored into the global | |
1059 | variable @code{yylval}, which is where the Bison parser will look for it. | |
1060 | (The C data type of @code{yylval} is @code{YYSTYPE}, which was defined | |
1061 | at the beginning of the grammar; @pxref{Rpcalc Decls, ,Declarations for @code{rpcalc}}.) | |
1062 | ||
1063 | A token type code of zero is returned if the end-of-file is encountered. | |
1064 | (Bison recognizes any nonpositive value as indicating the end of the | |
1065 | input.) | |
1066 | ||
1067 | Here is the code for the lexical analyzer: | |
1068 | ||
1069 | @example | |
1070 | @group | |
13863333 | 1071 | /* Lexical analyzer returns a double floating point |
bfa74976 RS |
1072 | number on the stack and the token NUM, or the ASCII |
1073 | character read if not a number. Skips all blanks | |
1074 | and tabs, returns 0 for EOF. */ | |
1075 | ||
1076 | #include <ctype.h> | |
1077 | @end group | |
1078 | ||
1079 | @group | |
13863333 AD |
1080 | int |
1081 | yylex (void) | |
bfa74976 RS |
1082 | @{ |
1083 | int c; | |
1084 | ||
1085 | /* skip white space */ | |
13863333 | 1086 | while ((c = getchar ()) == ' ' || c == '\t') |
bfa74976 RS |
1087 | ; |
1088 | @end group | |
1089 | @group | |
1090 | /* process numbers */ | |
13863333 | 1091 | if (c == '.' || isdigit (c)) |
bfa74976 RS |
1092 | @{ |
1093 | ungetc (c, stdin); | |
1094 | scanf ("%lf", &yylval); | |
1095 | return NUM; | |
1096 | @} | |
1097 | @end group | |
1098 | @group | |
1099 | /* return end-of-file */ | |
13863333 | 1100 | if (c == EOF) |
bfa74976 RS |
1101 | return 0; |
1102 | /* return single chars */ | |
13863333 | 1103 | return c; |
bfa74976 RS |
1104 | @} |
1105 | @end group | |
1106 | @end example | |
1107 | ||
1108 | @node Rpcalc Main, Rpcalc Error, Rpcalc Lexer, RPN Calc | |
1109 | @subsection The Controlling Function | |
1110 | @cindex controlling function | |
1111 | @cindex main function in simple example | |
1112 | ||
1113 | In keeping with the spirit of this example, the controlling function is | |
1114 | kept to the bare minimum. The only requirement is that it call | |
1115 | @code{yyparse} to start the process of parsing. | |
1116 | ||
1117 | @example | |
1118 | @group | |
13863333 AD |
1119 | int |
1120 | main (void) | |
bfa74976 | 1121 | @{ |
13863333 | 1122 | return yyparse (); |
bfa74976 RS |
1123 | @} |
1124 | @end group | |
1125 | @end example | |
1126 | ||
1127 | @node Rpcalc Error, Rpcalc Gen, Rpcalc Main, RPN Calc | |
1128 | @subsection The Error Reporting Routine | |
1129 | @cindex error reporting routine | |
1130 | ||
1131 | When @code{yyparse} detects a syntax error, it calls the error reporting | |
13863333 AD |
1132 | function @code{yyerror} to print an error message (usually but not |
1133 | always @code{"parse error"}). It is up to the programmer to supply | |
1134 | @code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so | |
1135 | here is the definition we will use: | |
bfa74976 RS |
1136 | |
1137 | @example | |
1138 | @group | |
1139 | #include <stdio.h> | |
1140 | ||
13863333 AD |
1141 | void |
1142 | yyerror (const char *s) /* Called by yyparse on error */ | |
bfa74976 RS |
1143 | @{ |
1144 | printf ("%s\n", s); | |
1145 | @} | |
1146 | @end group | |
1147 | @end example | |
1148 | ||
1149 | After @code{yyerror} returns, the Bison parser may recover from the error | |
1150 | and continue parsing if the grammar contains a suitable error rule | |
1151 | (@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We | |
1152 | have not written any error rules in this example, so any invalid input will | |
1153 | cause the calculator program to exit. This is not clean behavior for a | |
9ecbd125 | 1154 | real calculator, but it is adequate for the first example. |
bfa74976 RS |
1155 | |
1156 | @node Rpcalc Gen, Rpcalc Compile, Rpcalc Error, RPN Calc | |
1157 | @subsection Running Bison to Make the Parser | |
1158 | @cindex running Bison (introduction) | |
1159 | ||
ceed8467 AD |
1160 | Before running Bison to produce a parser, we need to decide how to |
1161 | arrange all the source code in one or more source files. For such a | |
1162 | simple example, the easiest thing is to put everything in one file. The | |
1163 | definitions of @code{yylex}, @code{yyerror} and @code{main} go at the | |
75f5aaea MA |
1164 | end, in the epilogue of the file |
1165 | (@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}). | |
bfa74976 RS |
1166 | |
1167 | For a large project, you would probably have several source files, and use | |
1168 | @code{make} to arrange to recompile them. | |
1169 | ||
1170 | With all the source in a single file, you use the following command to | |
1171 | convert it into a parser file: | |
1172 | ||
1173 | @example | |
1174 | bison @var{file_name}.y | |
1175 | @end example | |
1176 | ||
1177 | @noindent | |
1178 | In this example the file was called @file{rpcalc.y} (for ``Reverse Polish | |
1179 | CALCulator''). Bison produces a file named @file{@var{file_name}.tab.c}, | |
1180 | removing the @samp{.y} from the original file name. The file output by | |
1181 | Bison contains the source code for @code{yyparse}. The additional | |
1182 | functions in the input file (@code{yylex}, @code{yyerror} and @code{main}) | |
1183 | are copied verbatim to the output. | |
1184 | ||
1185 | @node Rpcalc Compile, , Rpcalc Gen, RPN Calc | |
1186 | @subsection Compiling the Parser File | |
1187 | @cindex compiling the parser | |
1188 | ||
1189 | Here is how to compile and run the parser file: | |
1190 | ||
1191 | @example | |
1192 | @group | |
1193 | # @r{List files in current directory.} | |
1194 | % ls | |
1195 | rpcalc.tab.c rpcalc.y | |
1196 | @end group | |
1197 | ||
1198 | @group | |
1199 | # @r{Compile the Bison parser.} | |
1200 | # @r{@samp{-lm} tells compiler to search math library for @code{pow}.} | |
1201 | % cc rpcalc.tab.c -lm -o rpcalc | |
1202 | @end group | |
1203 | ||
1204 | @group | |
1205 | # @r{List files again.} | |
1206 | % ls | |
1207 | rpcalc rpcalc.tab.c rpcalc.y | |
1208 | @end group | |
1209 | @end example | |
1210 | ||
1211 | The file @file{rpcalc} now contains the executable code. Here is an | |
1212 | example session using @code{rpcalc}. | |
1213 | ||
1214 | @example | |
1215 | % rpcalc | |
1216 | 4 9 + | |
1217 | 13 | |
1218 | 3 7 + 3 4 5 *+- | |
1219 | -13 | |
1220 | 3 7 + 3 4 5 * + - n @r{Note the unary minus, @samp{n}} | |
1221 | 13 | |
1222 | 5 6 / 4 n + | |
1223 | -3.166666667 | |
1224 | 3 4 ^ @r{Exponentiation} | |
1225 | 81 | |
1226 | ^D @r{End-of-file indicator} | |
1227 | % | |
1228 | @end example | |
1229 | ||
1230 | @node Infix Calc, Simple Error Recovery, RPN Calc, Examples | |
1231 | @section Infix Notation Calculator: @code{calc} | |
1232 | @cindex infix notation calculator | |
1233 | @cindex @code{calc} | |
1234 | @cindex calculator, infix notation | |
1235 | ||
1236 | We now modify rpcalc to handle infix operators instead of postfix. Infix | |
1237 | notation involves the concept of operator precedence and the need for | |
1238 | parentheses nested to arbitrary depth. Here is the Bison code for | |
1239 | @file{calc.y}, an infix desk-top calculator. | |
1240 | ||
1241 | @example | |
1242 | /* Infix notation calculator--calc */ | |
1243 | ||
1244 | %@{ | |
1245 | #define YYSTYPE double | |
1246 | #include <math.h> | |
1247 | %@} | |
1248 | ||
1249 | /* BISON Declarations */ | |
1250 | %token NUM | |
1251 | %left '-' '+' | |
1252 | %left '*' '/' | |
1253 | %left NEG /* negation--unary minus */ | |
1254 | %right '^' /* exponentiation */ | |
1255 | ||
1256 | /* Grammar follows */ | |
1257 | %% | |
1258 | input: /* empty string */ | |
1259 | | input line | |
1260 | ; | |
1261 | ||
1262 | line: '\n' | |
1263 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
1264 | ; | |
1265 | ||
1266 | exp: NUM @{ $$ = $1; @} | |
1267 | | exp '+' exp @{ $$ = $1 + $3; @} | |
1268 | | exp '-' exp @{ $$ = $1 - $3; @} | |
1269 | | exp '*' exp @{ $$ = $1 * $3; @} | |
1270 | | exp '/' exp @{ $$ = $1 / $3; @} | |
1271 | | '-' exp %prec NEG @{ $$ = -$2; @} | |
1272 | | exp '^' exp @{ $$ = pow ($1, $3); @} | |
1273 | | '(' exp ')' @{ $$ = $2; @} | |
1274 | ; | |
1275 | %% | |
1276 | @end example | |
1277 | ||
1278 | @noindent | |
ceed8467 AD |
1279 | The functions @code{yylex}, @code{yyerror} and @code{main} can be the |
1280 | same as before. | |
bfa74976 RS |
1281 | |
1282 | There are two important new features shown in this code. | |
1283 | ||
1284 | In the second section (Bison declarations), @code{%left} declares token | |
1285 | types and says they are left-associative operators. The declarations | |
1286 | @code{%left} and @code{%right} (right associativity) take the place of | |
1287 | @code{%token} which is used to declare a token type name without | |
1288 | associativity. (These tokens are single-character literals, which | |
1289 | ordinarily don't need to be declared. We declare them here to specify | |
1290 | the associativity.) | |
1291 | ||
1292 | Operator precedence is determined by the line ordering of the | |
1293 | declarations; the higher the line number of the declaration (lower on | |
1294 | the page or screen), the higher the precedence. Hence, exponentiation | |
1295 | has the highest precedence, unary minus (@code{NEG}) is next, followed | |
1296 | by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator Precedence}. | |
1297 | ||
1298 | The other important new feature is the @code{%prec} in the grammar section | |
1299 | for the unary minus operator. The @code{%prec} simply instructs Bison that | |
1300 | the rule @samp{| '-' exp} has the same precedence as @code{NEG}---in this | |
1301 | case the next-to-highest. @xref{Contextual Precedence, ,Context-Dependent Precedence}. | |
1302 | ||
1303 | Here is a sample run of @file{calc.y}: | |
1304 | ||
1305 | @need 500 | |
1306 | @example | |
1307 | % calc | |
1308 | 4 + 4.5 - (34/(8*3+-3)) | |
1309 | 6.880952381 | |
1310 | -56 + 2 | |
1311 | -54 | |
1312 | 3 ^ 2 | |
1313 | 9 | |
1314 | @end example | |
1315 | ||
1316 | @node Simple Error Recovery, Multi-function Calc, Infix Calc, Examples | |
1317 | @section Simple Error Recovery | |
1318 | @cindex error recovery, simple | |
1319 | ||
1320 | Up to this point, this manual has not addressed the issue of @dfn{error | |
1321 | recovery}---how to continue parsing after the parser detects a syntax | |
ceed8467 AD |
1322 | error. All we have handled is error reporting with @code{yyerror}. |
1323 | Recall that by default @code{yyparse} returns after calling | |
1324 | @code{yyerror}. This means that an erroneous input line causes the | |
1325 | calculator program to exit. Now we show how to rectify this deficiency. | |
bfa74976 RS |
1326 | |
1327 | The Bison language itself includes the reserved word @code{error}, which | |
1328 | may be included in the grammar rules. In the example below it has | |
1329 | been added to one of the alternatives for @code{line}: | |
1330 | ||
1331 | @example | |
1332 | @group | |
1333 | line: '\n' | |
1334 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
1335 | | error '\n' @{ yyerrok; @} | |
1336 | ; | |
1337 | @end group | |
1338 | @end example | |
1339 | ||
ceed8467 AD |
1340 | This addition to the grammar allows for simple error recovery in the |
1341 | event of a parse error. If an expression that cannot be evaluated is | |
1342 | read, the error will be recognized by the third rule for @code{line}, | |
1343 | and parsing will continue. (The @code{yyerror} function is still called | |
1344 | upon to print its message as well.) The action executes the statement | |
1345 | @code{yyerrok}, a macro defined automatically by Bison; its meaning is | |
1346 | that error recovery is complete (@pxref{Error Recovery}). Note the | |
1347 | difference between @code{yyerrok} and @code{yyerror}; neither one is a | |
1348 | misprint.@refill | |
bfa74976 RS |
1349 | |
1350 | This form of error recovery deals with syntax errors. There are other | |
1351 | kinds of errors; for example, division by zero, which raises an exception | |
1352 | signal that is normally fatal. A real calculator program must handle this | |
1353 | signal and use @code{longjmp} to return to @code{main} and resume parsing | |
1354 | input lines; it would also have to discard the rest of the current line of | |
1355 | input. We won't discuss this issue further because it is not specific to | |
1356 | Bison programs. | |
1357 | ||
1358 | @node Multi-function Calc, Exercises, Simple Error Recovery, Examples | |
1359 | @section Multi-Function Calculator: @code{mfcalc} | |
1360 | @cindex multi-function calculator | |
1361 | @cindex @code{mfcalc} | |
1362 | @cindex calculator, multi-function | |
1363 | ||
1364 | Now that the basics of Bison have been discussed, it is time to move on to | |
1365 | a more advanced problem. The above calculators provided only five | |
1366 | functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would | |
1367 | be nice to have a calculator that provides other mathematical functions such | |
1368 | as @code{sin}, @code{cos}, etc. | |
1369 | ||
1370 | It is easy to add new operators to the infix calculator as long as they are | |
1371 | only single-character literals. The lexical analyzer @code{yylex} passes | |
9ecbd125 | 1372 | back all nonnumber characters as tokens, so new grammar rules suffice for |
bfa74976 RS |
1373 | adding a new operator. But we want something more flexible: built-in |
1374 | functions whose syntax has this form: | |
1375 | ||
1376 | @example | |
1377 | @var{function_name} (@var{argument}) | |
1378 | @end example | |
1379 | ||
1380 | @noindent | |
1381 | At the same time, we will add memory to the calculator, by allowing you | |
1382 | to create named variables, store values in them, and use them later. | |
1383 | Here is a sample session with the multi-function calculator: | |
1384 | ||
1385 | @example | |
2a2e87db | 1386 | % mfcalc |
bfa74976 RS |
1387 | pi = 3.141592653589 |
1388 | 3.1415926536 | |
1389 | sin(pi) | |
1390 | 0.0000000000 | |
1391 | alpha = beta1 = 2.3 | |
1392 | 2.3000000000 | |
1393 | alpha | |
1394 | 2.3000000000 | |
1395 | ln(alpha) | |
1396 | 0.8329091229 | |
1397 | exp(ln(beta1)) | |
1398 | 2.3000000000 | |
1399 | % | |
1400 | @end example | |
1401 | ||
1402 | Note that multiple assignment and nested function calls are permitted. | |
1403 | ||
1404 | @menu | |
1405 | * Decl: Mfcalc Decl. Bison declarations for multi-function calculator. | |
1406 | * Rules: Mfcalc Rules. Grammar rules for the calculator. | |
1407 | * Symtab: Mfcalc Symtab. Symbol table management subroutines. | |
1408 | @end menu | |
1409 | ||
1410 | @node Mfcalc Decl, Mfcalc Rules, , Multi-function Calc | |
1411 | @subsection Declarations for @code{mfcalc} | |
1412 | ||
1413 | Here are the C and Bison declarations for the multi-function calculator. | |
1414 | ||
1415 | @smallexample | |
1416 | %@{ | |
1417 | #include <math.h> /* For math functions, cos(), sin(), etc. */ | |
1418 | #include "calc.h" /* Contains definition of `symrec' */ | |
1419 | %@} | |
1420 | %union @{ | |
1421 | double val; /* For returning numbers. */ | |
1422 | symrec *tptr; /* For returning symbol-table pointers */ | |
1423 | @} | |
1424 | ||
1425 | %token <val> NUM /* Simple double precision number */ | |
1426 | %token <tptr> VAR FNCT /* Variable and Function */ | |
1427 | %type <val> exp | |
1428 | ||
1429 | %right '=' | |
1430 | %left '-' '+' | |
1431 | %left '*' '/' | |
1432 | %left NEG /* Negation--unary minus */ | |
1433 | %right '^' /* Exponentiation */ | |
1434 | ||
1435 | /* Grammar follows */ | |
1436 | ||
1437 | %% | |
1438 | @end smallexample | |
1439 | ||
1440 | The above grammar introduces only two new features of the Bison language. | |
1441 | These features allow semantic values to have various data types | |
1442 | (@pxref{Multiple Types, ,More Than One Value Type}). | |
1443 | ||
1444 | The @code{%union} declaration specifies the entire list of possible types; | |
1445 | this is instead of defining @code{YYSTYPE}. The allowable types are now | |
1446 | double-floats (for @code{exp} and @code{NUM}) and pointers to entries in | |
1447 | the symbol table. @xref{Union Decl, ,The Collection of Value Types}. | |
1448 | ||
1449 | Since values can now have various types, it is necessary to associate a | |
1450 | type with each grammar symbol whose semantic value is used. These symbols | |
1451 | are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their | |
1452 | declarations are augmented with information about their data type (placed | |
1453 | between angle brackets). | |
1454 | ||
1455 | The Bison construct @code{%type} is used for declaring nonterminal symbols, | |
1456 | just as @code{%token} is used for declaring token types. We have not used | |
1457 | @code{%type} before because nonterminal symbols are normally declared | |
1458 | implicitly by the rules that define them. But @code{exp} must be declared | |
1459 | explicitly so we can specify its value type. @xref{Type Decl, ,Nonterminal Symbols}. | |
1460 | ||
1461 | @node Mfcalc Rules, Mfcalc Symtab, Mfcalc Decl, Multi-function Calc | |
1462 | @subsection Grammar Rules for @code{mfcalc} | |
1463 | ||
1464 | Here are the grammar rules for the multi-function calculator. | |
1465 | Most of them are copied directly from @code{calc}; three rules, | |
1466 | those which mention @code{VAR} or @code{FNCT}, are new. | |
1467 | ||
1468 | @smallexample | |
1469 | input: /* empty */ | |
1470 | | input line | |
1471 | ; | |
1472 | ||
1473 | line: | |
1474 | '\n' | |
1475 | | exp '\n' @{ printf ("\t%.10g\n", $1); @} | |
1476 | | error '\n' @{ yyerrok; @} | |
1477 | ; | |
1478 | ||
1479 | exp: NUM @{ $$ = $1; @} | |
1480 | | VAR @{ $$ = $1->value.var; @} | |
1481 | | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @} | |
1482 | | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @} | |
1483 | | exp '+' exp @{ $$ = $1 + $3; @} | |
1484 | | exp '-' exp @{ $$ = $1 - $3; @} | |
1485 | | exp '*' exp @{ $$ = $1 * $3; @} | |
1486 | | exp '/' exp @{ $$ = $1 / $3; @} | |
1487 | | '-' exp %prec NEG @{ $$ = -$2; @} | |
1488 | | exp '^' exp @{ $$ = pow ($1, $3); @} | |
1489 | | '(' exp ')' @{ $$ = $2; @} | |
1490 | ; | |
1491 | /* End of grammar */ | |
1492 | %% | |
1493 | @end smallexample | |
1494 | ||
1495 | @node Mfcalc Symtab, , Mfcalc Rules, Multi-function Calc | |
1496 | @subsection The @code{mfcalc} Symbol Table | |
1497 | @cindex symbol table example | |
1498 | ||
1499 | The multi-function calculator requires a symbol table to keep track of the | |
1500 | names and meanings of variables and functions. This doesn't affect the | |
1501 | grammar rules (except for the actions) or the Bison declarations, but it | |
1502 | requires some additional C functions for support. | |
1503 | ||
1504 | The symbol table itself consists of a linked list of records. Its | |
1505 | definition, which is kept in the header @file{calc.h}, is as follows. It | |
1506 | provides for either functions or variables to be placed in the table. | |
1507 | ||
1508 | @smallexample | |
1509 | @group | |
32dfccf8 AD |
1510 | /* Fonctions type. */ |
1511 | typedef double (*func_t) (double); | |
1512 | ||
bfa74976 RS |
1513 | /* Data type for links in the chain of symbols. */ |
1514 | struct symrec | |
1515 | @{ | |
1516 | char *name; /* name of symbol */ | |
1517 | int type; /* type of symbol: either VAR or FNCT */ | |
32dfccf8 AD |
1518 | union |
1519 | @{ | |
1520 | double var; /* value of a VAR */ | |
1521 | func_t fnctptr; /* value of a FNCT */ | |
bfa74976 RS |
1522 | @} value; |
1523 | struct symrec *next; /* link field */ | |
1524 | @}; | |
1525 | @end group | |
1526 | ||
1527 | @group | |
1528 | typedef struct symrec symrec; | |
1529 | ||
1530 | /* The symbol table: a chain of `struct symrec'. */ | |
1531 | extern symrec *sym_table; | |
1532 | ||
32dfccf8 AD |
1533 | symrec *putsym (const char *, func_t); |
1534 | symrec *getsym (const char *); | |
bfa74976 RS |
1535 | @end group |
1536 | @end smallexample | |
1537 | ||
1538 | The new version of @code{main} includes a call to @code{init_table}, a | |
1539 | function that initializes the symbol table. Here it is, and | |
1540 | @code{init_table} as well: | |
1541 | ||
1542 | @smallexample | |
1543 | @group | |
1544 | #include <stdio.h> | |
1545 | ||
13863333 AD |
1546 | int |
1547 | main (void) | |
bfa74976 RS |
1548 | @{ |
1549 | init_table (); | |
13863333 | 1550 | return yyparse (); |
bfa74976 RS |
1551 | @} |
1552 | @end group | |
1553 | ||
1554 | @group | |
13863333 AD |
1555 | void |
1556 | yyerror (const char *s) /* Called by yyparse on error */ | |
bfa74976 RS |
1557 | @{ |
1558 | printf ("%s\n", s); | |
1559 | @} | |
1560 | ||
1561 | struct init | |
1562 | @{ | |
1563 | char *fname; | |
32dfccf8 | 1564 | double (*fnct)(double); |
bfa74976 RS |
1565 | @}; |
1566 | @end group | |
1567 | ||
1568 | @group | |
13863333 AD |
1569 | struct init arith_fncts[] = |
1570 | @{ | |
32dfccf8 AD |
1571 | "sin", sin, |
1572 | "cos", cos, | |
13863333 | 1573 | "atan", atan, |
32dfccf8 AD |
1574 | "ln", log, |
1575 | "exp", exp, | |
13863333 AD |
1576 | "sqrt", sqrt, |
1577 | 0, 0 | |
1578 | @}; | |
bfa74976 RS |
1579 | |
1580 | /* The symbol table: a chain of `struct symrec'. */ | |
32dfccf8 | 1581 | symrec *sym_table = (symrec *) 0; |
bfa74976 RS |
1582 | @end group |
1583 | ||
1584 | @group | |
13863333 AD |
1585 | /* Put arithmetic functions in table. */ |
1586 | void | |
1587 | init_table (void) | |
bfa74976 RS |
1588 | @{ |
1589 | int i; | |
1590 | symrec *ptr; | |
1591 | for (i = 0; arith_fncts[i].fname != 0; i++) | |
1592 | @{ | |
1593 | ptr = putsym (arith_fncts[i].fname, FNCT); | |
1594 | ptr->value.fnctptr = arith_fncts[i].fnct; | |
1595 | @} | |
1596 | @} | |
1597 | @end group | |
1598 | @end smallexample | |
1599 | ||
1600 | By simply editing the initialization list and adding the necessary include | |
1601 | files, you can add additional functions to the calculator. | |
1602 | ||
1603 | Two important functions allow look-up and installation of symbols in the | |
1604 | symbol table. The function @code{putsym} is passed a name and the type | |
1605 | (@code{VAR} or @code{FNCT}) of the object to be installed. The object is | |
1606 | linked to the front of the list, and a pointer to the object is returned. | |
1607 | The function @code{getsym} is passed the name of the symbol to look up. If | |
1608 | found, a pointer to that symbol is returned; otherwise zero is returned. | |
1609 | ||
1610 | @smallexample | |
1611 | symrec * | |
13863333 | 1612 | putsym (char *sym_name, int sym_type) |
bfa74976 RS |
1613 | @{ |
1614 | symrec *ptr; | |
1615 | ptr = (symrec *) malloc (sizeof (symrec)); | |
1616 | ptr->name = (char *) malloc (strlen (sym_name) + 1); | |
1617 | strcpy (ptr->name,sym_name); | |
1618 | ptr->type = sym_type; | |
1619 | ptr->value.var = 0; /* set value to 0 even if fctn. */ | |
1620 | ptr->next = (struct symrec *)sym_table; | |
1621 | sym_table = ptr; | |
1622 | return ptr; | |
1623 | @} | |
1624 | ||
1625 | symrec * | |
13863333 | 1626 | getsym (const char *sym_name) |
bfa74976 RS |
1627 | @{ |
1628 | symrec *ptr; | |
1629 | for (ptr = sym_table; ptr != (symrec *) 0; | |
1630 | ptr = (symrec *)ptr->next) | |
1631 | if (strcmp (ptr->name,sym_name) == 0) | |
1632 | return ptr; | |
1633 | return 0; | |
1634 | @} | |
1635 | @end smallexample | |
1636 | ||
1637 | The function @code{yylex} must now recognize variables, numeric values, and | |
1638 | the single-character arithmetic operators. Strings of alphanumeric | |
14ded682 | 1639 | characters with a leading non-digit are recognized as either variables or |
bfa74976 RS |
1640 | functions depending on what the symbol table says about them. |
1641 | ||
1642 | The string is passed to @code{getsym} for look up in the symbol table. If | |
1643 | the name appears in the table, a pointer to its location and its type | |
1644 | (@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not | |
1645 | already in the table, then it is installed as a @code{VAR} using | |
1646 | @code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is | |
1647 | returned to @code{yyparse}.@refill | |
1648 | ||
1649 | No change is needed in the handling of numeric values and arithmetic | |
1650 | operators in @code{yylex}. | |
1651 | ||
1652 | @smallexample | |
1653 | @group | |
1654 | #include <ctype.h> | |
13863333 AD |
1655 | |
1656 | int | |
1657 | yylex (void) | |
bfa74976 RS |
1658 | @{ |
1659 | int c; | |
1660 | ||
1661 | /* Ignore whitespace, get first nonwhite character. */ | |
1662 | while ((c = getchar ()) == ' ' || c == '\t'); | |
1663 | ||
1664 | if (c == EOF) | |
1665 | return 0; | |
1666 | @end group | |
1667 | ||
1668 | @group | |
1669 | /* Char starts a number => parse the number. */ | |
1670 | if (c == '.' || isdigit (c)) | |
1671 | @{ | |
1672 | ungetc (c, stdin); | |
1673 | scanf ("%lf", &yylval.val); | |
1674 | return NUM; | |
1675 | @} | |
1676 | @end group | |
1677 | ||
1678 | @group | |
1679 | /* Char starts an identifier => read the name. */ | |
1680 | if (isalpha (c)) | |
1681 | @{ | |
1682 | symrec *s; | |
1683 | static char *symbuf = 0; | |
1684 | static int length = 0; | |
1685 | int i; | |
1686 | @end group | |
1687 | ||
1688 | @group | |
1689 | /* Initially make the buffer long enough | |
1690 | for a 40-character symbol name. */ | |
1691 | if (length == 0) | |
1692 | length = 40, symbuf = (char *)malloc (length + 1); | |
1693 | ||
1694 | i = 0; | |
1695 | do | |
1696 | @end group | |
1697 | @group | |
1698 | @{ | |
1699 | /* If buffer is full, make it bigger. */ | |
1700 | if (i == length) | |
1701 | @{ | |
1702 | length *= 2; | |
1703 | symbuf = (char *)realloc (symbuf, length + 1); | |
1704 | @} | |
1705 | /* Add this character to the buffer. */ | |
1706 | symbuf[i++] = c; | |
1707 | /* Get another character. */ | |
1708 | c = getchar (); | |
1709 | @} | |
1710 | @end group | |
1711 | @group | |
1712 | while (c != EOF && isalnum (c)); | |
1713 | ||
1714 | ungetc (c, stdin); | |
1715 | symbuf[i] = '\0'; | |
1716 | @end group | |
1717 | ||
1718 | @group | |
1719 | s = getsym (symbuf); | |
1720 | if (s == 0) | |
1721 | s = putsym (symbuf, VAR); | |
1722 | yylval.tptr = s; | |
1723 | return s->type; | |
1724 | @} | |
1725 | ||
1726 | /* Any other character is a token by itself. */ | |
1727 | return c; | |
1728 | @} | |
1729 | @end group | |
1730 | @end smallexample | |
1731 | ||
1732 | This program is both powerful and flexible. You may easily add new | |
1733 | functions, and it is a simple job to modify this code to install predefined | |
1734 | variables such as @code{pi} or @code{e} as well. | |
1735 | ||
1736 | @node Exercises, , Multi-function Calc, Examples | |
1737 | @section Exercises | |
1738 | @cindex exercises | |
1739 | ||
1740 | @enumerate | |
1741 | @item | |
1742 | Add some new functions from @file{math.h} to the initialization list. | |
1743 | ||
1744 | @item | |
1745 | Add another array that contains constants and their values. Then | |
1746 | modify @code{init_table} to add these constants to the symbol table. | |
1747 | It will be easiest to give the constants type @code{VAR}. | |
1748 | ||
1749 | @item | |
1750 | Make the program report an error if the user refers to an | |
1751 | uninitialized variable in any way except to store a value in it. | |
1752 | @end enumerate | |
1753 | ||
1754 | @node Grammar File, Interface, Examples, Top | |
1755 | @chapter Bison Grammar Files | |
1756 | ||
1757 | Bison takes as input a context-free grammar specification and produces a | |
1758 | C-language function that recognizes correct instances of the grammar. | |
1759 | ||
1760 | The Bison grammar input file conventionally has a name ending in @samp{.y}. | |
234a3be3 | 1761 | @xref{Invocation, ,Invoking Bison}. |
bfa74976 RS |
1762 | |
1763 | @menu | |
1764 | * Grammar Outline:: Overall layout of the grammar file. | |
1765 | * Symbols:: Terminal and nonterminal symbols. | |
1766 | * Rules:: How to write grammar rules. | |
1767 | * Recursion:: Writing recursive rules. | |
1768 | * Semantics:: Semantic values and actions. | |
847bf1f5 | 1769 | * Locations:: Locations and actions. |
bfa74976 RS |
1770 | * Declarations:: All kinds of Bison declarations are described here. |
1771 | * Multiple Parsers:: Putting more than one Bison parser in one program. | |
1772 | @end menu | |
1773 | ||
1774 | @node Grammar Outline, Symbols, , Grammar File | |
1775 | @section Outline of a Bison Grammar | |
1776 | ||
1777 | A Bison grammar file has four main sections, shown here with the | |
1778 | appropriate delimiters: | |
1779 | ||
1780 | @example | |
1781 | %@{ | |
75f5aaea | 1782 | @var{Prologue} |
bfa74976 RS |
1783 | %@} |
1784 | ||
1785 | @var{Bison declarations} | |
1786 | ||
1787 | %% | |
1788 | @var{Grammar rules} | |
1789 | %% | |
1790 | ||
75f5aaea | 1791 | @var{Epilogue} |
bfa74976 RS |
1792 | @end example |
1793 | ||
1794 | Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections. | |
1795 | ||
1796 | @menu | |
75f5aaea | 1797 | * Prologue:: Syntax and usage of the prologue. |
bfa74976 RS |
1798 | * Bison Declarations:: Syntax and usage of the Bison declarations section. |
1799 | * Grammar Rules:: Syntax and usage of the grammar rules section. | |
75f5aaea | 1800 | * Epilogue:: Syntax and usage of the epilogue. |
bfa74976 RS |
1801 | @end menu |
1802 | ||
75f5aaea MA |
1803 | @node Prologue, Bison Declarations, , Grammar Outline |
1804 | @subsection The prologue | |
1805 | @cindex declarations section | |
1806 | @cindex Prologue | |
1807 | @cindex declarations | |
bfa74976 | 1808 | |
75f5aaea | 1809 | The @var{prologue} section contains macro definitions and |
bfa74976 RS |
1810 | declarations of functions and variables that are used in the actions in the |
1811 | grammar rules. These are copied to the beginning of the parser file so | |
1812 | that they precede the definition of @code{yyparse}. You can use | |
1813 | @samp{#include} to get the declarations from a header file. If you don't | |
1814 | need any C declarations, you may omit the @samp{%@{} and @samp{%@}} | |
1815 | delimiters that bracket this section. | |
1816 | ||
75f5aaea | 1817 | @node Bison Declarations, Grammar Rules, Prologue, Grammar Outline |
bfa74976 RS |
1818 | @subsection The Bison Declarations Section |
1819 | @cindex Bison declarations (introduction) | |
1820 | @cindex declarations, Bison (introduction) | |
1821 | ||
1822 | The @var{Bison declarations} section contains declarations that define | |
1823 | terminal and nonterminal symbols, specify precedence, and so on. | |
1824 | In some simple grammars you may not need any declarations. | |
1825 | @xref{Declarations, ,Bison Declarations}. | |
1826 | ||
75f5aaea | 1827 | @node Grammar Rules, Epilogue, Bison Declarations, Grammar Outline |
bfa74976 RS |
1828 | @subsection The Grammar Rules Section |
1829 | @cindex grammar rules section | |
1830 | @cindex rules section for grammar | |
1831 | ||
1832 | The @dfn{grammar rules} section contains one or more Bison grammar | |
1833 | rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}. | |
1834 | ||
1835 | There must always be at least one grammar rule, and the first | |
1836 | @samp{%%} (which precedes the grammar rules) may never be omitted even | |
1837 | if it is the first thing in the file. | |
1838 | ||
75f5aaea MA |
1839 | @node Epilogue, , Grammar Rules, Grammar Outline |
1840 | @subsection The epilogue | |
bfa74976 | 1841 | @cindex additional C code section |
75f5aaea | 1842 | @cindex epilogue |
bfa74976 RS |
1843 | @cindex C code, section for additional |
1844 | ||
75f5aaea MA |
1845 | The @var{epilogue} is copied verbatim to the end of the parser file, just as |
1846 | the @var{prologue} is copied to the beginning. This is the most convenient | |
1847 | place to put anything that you want to have in the parser file but which need | |
1848 | not come before the definition of @code{yyparse}. For example, the | |
1849 | definitions of @code{yylex} and @code{yyerror} often go here. | |
1850 | @xref{Interface, ,Parser C-Language Interface}. | |
bfa74976 RS |
1851 | |
1852 | If the last section is empty, you may omit the @samp{%%} that separates it | |
1853 | from the grammar rules. | |
1854 | ||
1855 | The Bison parser itself contains many static variables whose names start | |
1856 | with @samp{yy} and many macros whose names start with @samp{YY}. It is a | |
1857 | good idea to avoid using any such names (except those documented in this | |
75f5aaea | 1858 | manual) in the epilogue of the grammar file. |
bfa74976 RS |
1859 | |
1860 | @node Symbols, Rules, Grammar Outline, Grammar File | |
1861 | @section Symbols, Terminal and Nonterminal | |
1862 | @cindex nonterminal symbol | |
1863 | @cindex terminal symbol | |
1864 | @cindex token type | |
1865 | @cindex symbol | |
1866 | ||
1867 | @dfn{Symbols} in Bison grammars represent the grammatical classifications | |
1868 | of the language. | |
1869 | ||
1870 | A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a | |
1871 | class of syntactically equivalent tokens. You use the symbol in grammar | |
1872 | rules to mean that a token in that class is allowed. The symbol is | |
1873 | represented in the Bison parser by a numeric code, and the @code{yylex} | |
1874 | function returns a token type code to indicate what kind of token has been | |
1875 | read. You don't need to know what the code value is; you can use the | |
1876 | symbol to stand for it. | |
1877 | ||
1878 | A @dfn{nonterminal symbol} stands for a class of syntactically equivalent | |
1879 | groupings. The symbol name is used in writing grammar rules. By convention, | |
1880 | it should be all lower case. | |
1881 | ||
1882 | Symbol names can contain letters, digits (not at the beginning), | |
1883 | underscores and periods. Periods make sense only in nonterminals. | |
1884 | ||
931c7513 | 1885 | There are three ways of writing terminal symbols in the grammar: |
bfa74976 RS |
1886 | |
1887 | @itemize @bullet | |
1888 | @item | |
1889 | A @dfn{named token type} is written with an identifier, like an | |
1890 | identifier in C. By convention, it should be all upper case. Each | |
1891 | such name must be defined with a Bison declaration such as | |
1892 | @code{%token}. @xref{Token Decl, ,Token Type Names}. | |
1893 | ||
1894 | @item | |
1895 | @cindex character token | |
1896 | @cindex literal token | |
1897 | @cindex single-character literal | |
931c7513 RS |
1898 | A @dfn{character token type} (or @dfn{literal character token}) is |
1899 | written in the grammar using the same syntax used in C for character | |
1900 | constants; for example, @code{'+'} is a character token type. A | |
1901 | character token type doesn't need to be declared unless you need to | |
1902 | specify its semantic value data type (@pxref{Value Type, ,Data Types of | |
1903 | Semantic Values}), associativity, or precedence (@pxref{Precedence, | |
1904 | ,Operator Precedence}). | |
bfa74976 RS |
1905 | |
1906 | By convention, a character token type is used only to represent a | |
1907 | token that consists of that particular character. Thus, the token | |
1908 | type @code{'+'} is used to represent the character @samp{+} as a | |
1909 | token. Nothing enforces this convention, but if you depart from it, | |
1910 | your program will confuse other readers. | |
1911 | ||
1912 | All the usual escape sequences used in character literals in C can be | |
1913 | used in Bison as well, but you must not use the null character as a | |
931c7513 RS |
1914 | character literal because its ASCII code, zero, is the code @code{yylex} |
1915 | returns for end-of-input (@pxref{Calling Convention, ,Calling Convention | |
1916 | for @code{yylex}}). | |
1917 | ||
1918 | @item | |
1919 | @cindex string token | |
1920 | @cindex literal string token | |
9ecbd125 | 1921 | @cindex multicharacter literal |
931c7513 RS |
1922 | A @dfn{literal string token} is written like a C string constant; for |
1923 | example, @code{"<="} is a literal string token. A literal string token | |
1924 | doesn't need to be declared unless you need to specify its semantic | |
14ded682 | 1925 | value data type (@pxref{Value Type}), associativity, or precedence |
931c7513 RS |
1926 | (@pxref{Precedence}). |
1927 | ||
1928 | You can associate the literal string token with a symbolic name as an | |
1929 | alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token | |
1930 | Declarations}). If you don't do that, the lexical analyzer has to | |
1931 | retrieve the token number for the literal string token from the | |
1932 | @code{yytname} table (@pxref{Calling Convention}). | |
1933 | ||
1934 | @strong{WARNING}: literal string tokens do not work in Yacc. | |
1935 | ||
1936 | By convention, a literal string token is used only to represent a token | |
1937 | that consists of that particular string. Thus, you should use the token | |
1938 | type @code{"<="} to represent the string @samp{<=} as a token. Bison | |
9ecbd125 | 1939 | does not enforce this convention, but if you depart from it, people who |
931c7513 RS |
1940 | read your program will be confused. |
1941 | ||
1942 | All the escape sequences used in string literals in C can be used in | |
1943 | Bison as well. A literal string token must contain two or more | |
1944 | characters; for a token containing just one character, use a character | |
1945 | token (see above). | |
bfa74976 RS |
1946 | @end itemize |
1947 | ||
1948 | How you choose to write a terminal symbol has no effect on its | |
1949 | grammatical meaning. That depends only on where it appears in rules and | |
1950 | on when the parser function returns that symbol. | |
1951 | ||
1952 | The value returned by @code{yylex} is always one of the terminal symbols | |
1953 | (or 0 for end-of-input). Whichever way you write the token type in the | |
1954 | grammar rules, you write it the same way in the definition of @code{yylex}. | |
1955 | The numeric code for a character token type is simply the ASCII code for | |
1956 | the character, so @code{yylex} can use the identical character constant to | |
1957 | generate the requisite code. Each named token type becomes a C macro in | |
1958 | the parser file, so @code{yylex} can use the name to stand for the code. | |
13863333 | 1959 | (This is why periods don't make sense in terminal symbols.) |
bfa74976 RS |
1960 | @xref{Calling Convention, ,Calling Convention for @code{yylex}}. |
1961 | ||
1962 | If @code{yylex} is defined in a separate file, you need to arrange for the | |
1963 | token-type macro definitions to be available there. Use the @samp{-d} | |
1964 | option when you run Bison, so that it will write these macro definitions | |
1965 | into a separate header file @file{@var{name}.tab.h} which you can include | |
1966 | in the other source files that need it. @xref{Invocation, ,Invoking Bison}. | |
1967 | ||
1968 | The symbol @code{error} is a terminal symbol reserved for error recovery | |
1969 | (@pxref{Error Recovery}); you shouldn't use it for any other purpose. | |
1970 | In particular, @code{yylex} should never return this value. | |
1971 | ||
1972 | @node Rules, Recursion, Symbols, Grammar File | |
1973 | @section Syntax of Grammar Rules | |
1974 | @cindex rule syntax | |
1975 | @cindex grammar rule syntax | |
1976 | @cindex syntax of grammar rules | |
1977 | ||
1978 | A Bison grammar rule has the following general form: | |
1979 | ||
1980 | @example | |
e425e872 | 1981 | @group |
bfa74976 RS |
1982 | @var{result}: @var{components}@dots{} |
1983 | ; | |
e425e872 | 1984 | @end group |
bfa74976 RS |
1985 | @end example |
1986 | ||
1987 | @noindent | |
9ecbd125 | 1988 | where @var{result} is the nonterminal symbol that this rule describes, |
bfa74976 | 1989 | and @var{components} are various terminal and nonterminal symbols that |
13863333 | 1990 | are put together by this rule (@pxref{Symbols}). |
bfa74976 RS |
1991 | |
1992 | For example, | |
1993 | ||
1994 | @example | |
1995 | @group | |
1996 | exp: exp '+' exp | |
1997 | ; | |
1998 | @end group | |
1999 | @end example | |
2000 | ||
2001 | @noindent | |
2002 | says that two groupings of type @code{exp}, with a @samp{+} token in between, | |
2003 | can be combined into a larger grouping of type @code{exp}. | |
2004 | ||
2005 | Whitespace in rules is significant only to separate symbols. You can add | |
2006 | extra whitespace as you wish. | |
2007 | ||
2008 | Scattered among the components can be @var{actions} that determine | |
2009 | the semantics of the rule. An action looks like this: | |
2010 | ||
2011 | @example | |
2012 | @{@var{C statements}@} | |
2013 | @end example | |
2014 | ||
2015 | @noindent | |
2016 | Usually there is only one action and it follows the components. | |
2017 | @xref{Actions}. | |
2018 | ||
2019 | @findex | | |
2020 | Multiple rules for the same @var{result} can be written separately or can | |
2021 | be joined with the vertical-bar character @samp{|} as follows: | |
2022 | ||
2023 | @ifinfo | |
2024 | @example | |
2025 | @var{result}: @var{rule1-components}@dots{} | |
2026 | | @var{rule2-components}@dots{} | |
2027 | @dots{} | |
2028 | ; | |
2029 | @end example | |
2030 | @end ifinfo | |
2031 | @iftex | |
2032 | @example | |
2033 | @group | |
2034 | @var{result}: @var{rule1-components}@dots{} | |
2035 | | @var{rule2-components}@dots{} | |
2036 | @dots{} | |
2037 | ; | |
2038 | @end group | |
2039 | @end example | |
2040 | @end iftex | |
2041 | ||
2042 | @noindent | |
2043 | They are still considered distinct rules even when joined in this way. | |
2044 | ||
2045 | If @var{components} in a rule is empty, it means that @var{result} can | |
2046 | match the empty string. For example, here is how to define a | |
2047 | comma-separated sequence of zero or more @code{exp} groupings: | |
2048 | ||
2049 | @example | |
2050 | @group | |
2051 | expseq: /* empty */ | |
2052 | | expseq1 | |
2053 | ; | |
2054 | @end group | |
2055 | ||
2056 | @group | |
2057 | expseq1: exp | |
2058 | | expseq1 ',' exp | |
2059 | ; | |
2060 | @end group | |
2061 | @end example | |
2062 | ||
2063 | @noindent | |
2064 | It is customary to write a comment @samp{/* empty */} in each rule | |
2065 | with no components. | |
2066 | ||
2067 | @node Recursion, Semantics, Rules, Grammar File | |
2068 | @section Recursive Rules | |
2069 | @cindex recursive rule | |
2070 | ||
2071 | A rule is called @dfn{recursive} when its @var{result} nonterminal appears | |
2072 | also on its right hand side. Nearly all Bison grammars need to use | |
2073 | recursion, because that is the only way to define a sequence of any number | |
9ecbd125 JT |
2074 | of a particular thing. Consider this recursive definition of a |
2075 | comma-separated sequence of one or more expressions: | |
bfa74976 RS |
2076 | |
2077 | @example | |
2078 | @group | |
2079 | expseq1: exp | |
2080 | | expseq1 ',' exp | |
2081 | ; | |
2082 | @end group | |
2083 | @end example | |
2084 | ||
2085 | @cindex left recursion | |
2086 | @cindex right recursion | |
2087 | @noindent | |
2088 | Since the recursive use of @code{expseq1} is the leftmost symbol in the | |
2089 | right hand side, we call this @dfn{left recursion}. By contrast, here | |
2090 | the same construct is defined using @dfn{right recursion}: | |
2091 | ||
2092 | @example | |
2093 | @group | |
2094 | expseq1: exp | |
2095 | | exp ',' expseq1 | |
2096 | ; | |
2097 | @end group | |
2098 | @end example | |
2099 | ||
2100 | @noindent | |
2101 | Any kind of sequence can be defined using either left recursion or | |
2102 | right recursion, but you should always use left recursion, because it | |
2103 | can parse a sequence of any number of elements with bounded stack | |
2104 | space. Right recursion uses up space on the Bison stack in proportion | |
2105 | to the number of elements in the sequence, because all the elements | |
2106 | must be shifted onto the stack before the rule can be applied even | |
2107 | once. @xref{Algorithm, ,The Bison Parser Algorithm }, for | |
2108 | further explanation of this. | |
2109 | ||
2110 | @cindex mutual recursion | |
2111 | @dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the | |
2112 | rule does not appear directly on its right hand side, but does appear | |
2113 | in rules for other nonterminals which do appear on its right hand | |
13863333 | 2114 | side. |
bfa74976 RS |
2115 | |
2116 | For example: | |
2117 | ||
2118 | @example | |
2119 | @group | |
2120 | expr: primary | |
2121 | | primary '+' primary | |
2122 | ; | |
2123 | @end group | |
2124 | ||
2125 | @group | |
2126 | primary: constant | |
2127 | | '(' expr ')' | |
2128 | ; | |
2129 | @end group | |
2130 | @end example | |
2131 | ||
2132 | @noindent | |
2133 | defines two mutually-recursive nonterminals, since each refers to the | |
2134 | other. | |
2135 | ||
847bf1f5 | 2136 | @node Semantics, Locations, Recursion, Grammar File |
bfa74976 RS |
2137 | @section Defining Language Semantics |
2138 | @cindex defining language semantics | |
13863333 | 2139 | @cindex language semantics, defining |
bfa74976 RS |
2140 | |
2141 | The grammar rules for a language determine only the syntax. The semantics | |
2142 | are determined by the semantic values associated with various tokens and | |
2143 | groupings, and by the actions taken when various groupings are recognized. | |
2144 | ||
2145 | For example, the calculator calculates properly because the value | |
2146 | associated with each expression is the proper number; it adds properly | |
2147 | because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add | |
2148 | the numbers associated with @var{x} and @var{y}. | |
2149 | ||
2150 | @menu | |
2151 | * Value Type:: Specifying one data type for all semantic values. | |
2152 | * Multiple Types:: Specifying several alternative data types. | |
2153 | * Actions:: An action is the semantic definition of a grammar rule. | |
2154 | * Action Types:: Specifying data types for actions to operate on. | |
2155 | * Mid-Rule Actions:: Most actions go at the end of a rule. | |
2156 | This says when, why and how to use the exceptional | |
2157 | action in the middle of a rule. | |
2158 | @end menu | |
2159 | ||
2160 | @node Value Type, Multiple Types, , Semantics | |
2161 | @subsection Data Types of Semantic Values | |
2162 | @cindex semantic value type | |
2163 | @cindex value type, semantic | |
2164 | @cindex data types of semantic values | |
2165 | @cindex default data type | |
2166 | ||
2167 | In a simple program it may be sufficient to use the same data type for | |
2168 | the semantic values of all language constructs. This was true in the | |
2169 | RPN and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish Notation Calculator}). | |
2170 | ||
2171 | Bison's default is to use type @code{int} for all semantic values. To | |
2172 | specify some other type, define @code{YYSTYPE} as a macro, like this: | |
2173 | ||
2174 | @example | |
2175 | #define YYSTYPE double | |
2176 | @end example | |
2177 | ||
2178 | @noindent | |
75f5aaea MA |
2179 | This macro definition must go in the prologue of the grammar file |
2180 | (@pxref{Grammar Outline, ,Outline of a Bison Grammar}). | |
bfa74976 RS |
2181 | |
2182 | @node Multiple Types, Actions, Value Type, Semantics | |
2183 | @subsection More Than One Value Type | |
2184 | ||
2185 | In most programs, you will need different data types for different kinds | |
2186 | of tokens and groupings. For example, a numeric constant may need type | |
2187 | @code{int} or @code{long}, while a string constant needs type @code{char *}, | |
2188 | and an identifier might need a pointer to an entry in the symbol table. | |
2189 | ||
2190 | To use more than one data type for semantic values in one parser, Bison | |
2191 | requires you to do two things: | |
2192 | ||
2193 | @itemize @bullet | |
2194 | @item | |
2195 | Specify the entire collection of possible data types, with the | |
2196 | @code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of Value Types}). | |
2197 | ||
2198 | @item | |
14ded682 AD |
2199 | Choose one of those types for each symbol (terminal or nonterminal) for |
2200 | which semantic values are used. This is done for tokens with the | |
2201 | @code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names}) | |
2202 | and for groupings with the @code{%type} Bison declaration (@pxref{Type | |
2203 | Decl, ,Nonterminal Symbols}). | |
bfa74976 RS |
2204 | @end itemize |
2205 | ||
2206 | @node Actions, Action Types, Multiple Types, Semantics | |
2207 | @subsection Actions | |
2208 | @cindex action | |
2209 | @vindex $$ | |
2210 | @vindex $@var{n} | |
2211 | ||
2212 | An action accompanies a syntactic rule and contains C code to be executed | |
2213 | each time an instance of that rule is recognized. The task of most actions | |
2214 | is to compute a semantic value for the grouping built by the rule from the | |
2215 | semantic values associated with tokens or smaller groupings. | |
2216 | ||
2217 | An action consists of C statements surrounded by braces, much like a | |
2218 | compound statement in C. It can be placed at any position in the rule; it | |
2219 | is executed at that position. Most rules have just one action at the end | |
2220 | of the rule, following all the components. Actions in the middle of a rule | |
2221 | are tricky and used only for special purposes (@pxref{Mid-Rule Actions, ,Actions in Mid-Rule}). | |
2222 | ||
2223 | The C code in an action can refer to the semantic values of the components | |
2224 | matched by the rule with the construct @code{$@var{n}}, which stands for | |
2225 | the value of the @var{n}th component. The semantic value for the grouping | |
2226 | being constructed is @code{$$}. (Bison translates both of these constructs | |
2227 | into array element references when it copies the actions into the parser | |
2228 | file.) | |
2229 | ||
2230 | Here is a typical example: | |
2231 | ||
2232 | @example | |
2233 | @group | |
2234 | exp: @dots{} | |
2235 | | exp '+' exp | |
2236 | @{ $$ = $1 + $3; @} | |
2237 | @end group | |
2238 | @end example | |
2239 | ||
2240 | @noindent | |
2241 | This rule constructs an @code{exp} from two smaller @code{exp} groupings | |
2242 | connected by a plus-sign token. In the action, @code{$1} and @code{$3} | |
2243 | refer to the semantic values of the two component @code{exp} groupings, | |
2244 | which are the first and third symbols on the right hand side of the rule. | |
2245 | The sum is stored into @code{$$} so that it becomes the semantic value of | |
2246 | the addition-expression just recognized by the rule. If there were a | |
2247 | useful semantic value associated with the @samp{+} token, it could be | |
2248 | referred to as @code{$2}.@refill | |
2249 | ||
2250 | @cindex default action | |
2251 | If you don't specify an action for a rule, Bison supplies a default: | |
2252 | @w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule becomes | |
2253 | the value of the whole rule. Of course, the default rule is valid only | |
2254 | if the two data types match. There is no meaningful default action for | |
2255 | an empty rule; every empty rule must have an explicit action unless the | |
2256 | rule's value does not matter. | |
2257 | ||
2258 | @code{$@var{n}} with @var{n} zero or negative is allowed for reference | |
2259 | to tokens and groupings on the stack @emph{before} those that match the | |
2260 | current rule. This is a very risky practice, and to use it reliably | |
2261 | you must be certain of the context in which the rule is applied. Here | |
2262 | is a case in which you can use this reliably: | |
2263 | ||
2264 | @example | |
2265 | @group | |
2266 | foo: expr bar '+' expr @{ @dots{} @} | |
2267 | | expr bar '-' expr @{ @dots{} @} | |
2268 | ; | |
2269 | @end group | |
2270 | ||
2271 | @group | |
2272 | bar: /* empty */ | |
2273 | @{ previous_expr = $0; @} | |
2274 | ; | |
2275 | @end group | |
2276 | @end example | |
2277 | ||
2278 | As long as @code{bar} is used only in the fashion shown here, @code{$0} | |
2279 | always refers to the @code{expr} which precedes @code{bar} in the | |
2280 | definition of @code{foo}. | |
2281 | ||
2282 | @node Action Types, Mid-Rule Actions, Actions, Semantics | |
2283 | @subsection Data Types of Values in Actions | |
2284 | @cindex action data types | |
2285 | @cindex data types in actions | |
2286 | ||
2287 | If you have chosen a single data type for semantic values, the @code{$$} | |
2288 | and @code{$@var{n}} constructs always have that data type. | |
2289 | ||
2290 | If you have used @code{%union} to specify a variety of data types, then you | |
2291 | must declare a choice among these types for each terminal or nonterminal | |
2292 | symbol that can have a semantic value. Then each time you use @code{$$} or | |
2293 | @code{$@var{n}}, its data type is determined by which symbol it refers to | |
2294 | in the rule. In this example,@refill | |
2295 | ||
2296 | @example | |
2297 | @group | |
2298 | exp: @dots{} | |
2299 | | exp '+' exp | |
2300 | @{ $$ = $1 + $3; @} | |
2301 | @end group | |
2302 | @end example | |
2303 | ||
2304 | @noindent | |
2305 | @code{$1} and @code{$3} refer to instances of @code{exp}, so they all | |
2306 | have the data type declared for the nonterminal symbol @code{exp}. If | |
2307 | @code{$2} were used, it would have the data type declared for the | |
2308 | terminal symbol @code{'+'}, whatever that might be.@refill | |
2309 | ||
2310 | Alternatively, you can specify the data type when you refer to the value, | |
2311 | by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the | |
2312 | reference. For example, if you have defined types as shown here: | |
2313 | ||
2314 | @example | |
2315 | @group | |
2316 | %union @{ | |
2317 | int itype; | |
2318 | double dtype; | |
2319 | @} | |
2320 | @end group | |
2321 | @end example | |
2322 | ||
2323 | @noindent | |
2324 | then you can write @code{$<itype>1} to refer to the first subunit of the | |
2325 | rule as an integer, or @code{$<dtype>1} to refer to it as a double. | |
2326 | ||
2327 | @node Mid-Rule Actions, , Action Types, Semantics | |
2328 | @subsection Actions in Mid-Rule | |
2329 | @cindex actions in mid-rule | |
2330 | @cindex mid-rule actions | |
2331 | ||
2332 | Occasionally it is useful to put an action in the middle of a rule. | |
2333 | These actions are written just like usual end-of-rule actions, but they | |
2334 | are executed before the parser even recognizes the following components. | |
2335 | ||
2336 | A mid-rule action may refer to the components preceding it using | |
2337 | @code{$@var{n}}, but it may not refer to subsequent components because | |
2338 | it is run before they are parsed. | |
2339 | ||
2340 | The mid-rule action itself counts as one of the components of the rule. | |
2341 | This makes a difference when there is another action later in the same rule | |
2342 | (and usually there is another at the end): you have to count the actions | |
2343 | along with the symbols when working out which number @var{n} to use in | |
2344 | @code{$@var{n}}. | |
2345 | ||
2346 | The mid-rule action can also have a semantic value. The action can set | |
2347 | its value with an assignment to @code{$$}, and actions later in the rule | |
2348 | can refer to the value using @code{$@var{n}}. Since there is no symbol | |
2349 | to name the action, there is no way to declare a data type for the value | |
fdc6758b MA |
2350 | in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to |
2351 | specify a data type each time you refer to this value. | |
bfa74976 RS |
2352 | |
2353 | There is no way to set the value of the entire rule with a mid-rule | |
2354 | action, because assignments to @code{$$} do not have that effect. The | |
2355 | only way to set the value for the entire rule is with an ordinary action | |
2356 | at the end of the rule. | |
2357 | ||
2358 | Here is an example from a hypothetical compiler, handling a @code{let} | |
2359 | statement that looks like @samp{let (@var{variable}) @var{statement}} and | |
2360 | serves to create a variable named @var{variable} temporarily for the | |
2361 | duration of @var{statement}. To parse this construct, we must put | |
2362 | @var{variable} into the symbol table while @var{statement} is parsed, then | |
2363 | remove it afterward. Here is how it is done: | |
2364 | ||
2365 | @example | |
2366 | @group | |
2367 | stmt: LET '(' var ')' | |
2368 | @{ $<context>$ = push_context (); | |
2369 | declare_variable ($3); @} | |
2370 | stmt @{ $$ = $6; | |
2371 | pop_context ($<context>5); @} | |
2372 | @end group | |
2373 | @end example | |
2374 | ||
2375 | @noindent | |
2376 | As soon as @samp{let (@var{variable})} has been recognized, the first | |
2377 | action is run. It saves a copy of the current semantic context (the | |
2378 | list of accessible variables) as its semantic value, using alternative | |
2379 | @code{context} in the data-type union. Then it calls | |
2380 | @code{declare_variable} to add the new variable to that list. Once the | |
2381 | first action is finished, the embedded statement @code{stmt} can be | |
2382 | parsed. Note that the mid-rule action is component number 5, so the | |
2383 | @samp{stmt} is component number 6. | |
2384 | ||
2385 | After the embedded statement is parsed, its semantic value becomes the | |
2386 | value of the entire @code{let}-statement. Then the semantic value from the | |
2387 | earlier action is used to restore the prior list of variables. This | |
2388 | removes the temporary @code{let}-variable from the list so that it won't | |
2389 | appear to exist while the rest of the program is parsed. | |
2390 | ||
2391 | Taking action before a rule is completely recognized often leads to | |
2392 | conflicts since the parser must commit to a parse in order to execute the | |
2393 | action. For example, the following two rules, without mid-rule actions, | |
2394 | can coexist in a working parser because the parser can shift the open-brace | |
2395 | token and look at what follows before deciding whether there is a | |
2396 | declaration or not: | |
2397 | ||
2398 | @example | |
2399 | @group | |
2400 | compound: '@{' declarations statements '@}' | |
2401 | | '@{' statements '@}' | |
2402 | ; | |
2403 | @end group | |
2404 | @end example | |
2405 | ||
2406 | @noindent | |
2407 | But when we add a mid-rule action as follows, the rules become nonfunctional: | |
2408 | ||
2409 | @example | |
2410 | @group | |
2411 | compound: @{ prepare_for_local_variables (); @} | |
2412 | '@{' declarations statements '@}' | |
2413 | @end group | |
2414 | @group | |
2415 | | '@{' statements '@}' | |
2416 | ; | |
2417 | @end group | |
2418 | @end example | |
2419 | ||
2420 | @noindent | |
2421 | Now the parser is forced to decide whether to run the mid-rule action | |
2422 | when it has read no farther than the open-brace. In other words, it | |
2423 | must commit to using one rule or the other, without sufficient | |
2424 | information to do it correctly. (The open-brace token is what is called | |
2425 | the @dfn{look-ahead} token at this time, since the parser is still | |
2426 | deciding what to do about it. @xref{Look-Ahead, ,Look-Ahead Tokens}.) | |
2427 | ||
2428 | You might think that you could correct the problem by putting identical | |
2429 | actions into the two rules, like this: | |
2430 | ||
2431 | @example | |
2432 | @group | |
2433 | compound: @{ prepare_for_local_variables (); @} | |
2434 | '@{' declarations statements '@}' | |
2435 | | @{ prepare_for_local_variables (); @} | |
2436 | '@{' statements '@}' | |
2437 | ; | |
2438 | @end group | |
2439 | @end example | |
2440 | ||
2441 | @noindent | |
2442 | But this does not help, because Bison does not realize that the two actions | |
2443 | are identical. (Bison never tries to understand the C code in an action.) | |
2444 | ||
2445 | If the grammar is such that a declaration can be distinguished from a | |
2446 | statement by the first token (which is true in C), then one solution which | |
2447 | does work is to put the action after the open-brace, like this: | |
2448 | ||
2449 | @example | |
2450 | @group | |
2451 | compound: '@{' @{ prepare_for_local_variables (); @} | |
2452 | declarations statements '@}' | |
2453 | | '@{' statements '@}' | |
2454 | ; | |
2455 | @end group | |
2456 | @end example | |
2457 | ||
2458 | @noindent | |
2459 | Now the first token of the following declaration or statement, | |
2460 | which would in any case tell Bison which rule to use, can still do so. | |
2461 | ||
2462 | Another solution is to bury the action inside a nonterminal symbol which | |
2463 | serves as a subroutine: | |
2464 | ||
2465 | @example | |
2466 | @group | |
2467 | subroutine: /* empty */ | |
2468 | @{ prepare_for_local_variables (); @} | |
2469 | ; | |
2470 | ||
2471 | @end group | |
2472 | ||
2473 | @group | |
2474 | compound: subroutine | |
2475 | '@{' declarations statements '@}' | |
2476 | | subroutine | |
2477 | '@{' statements '@}' | |
2478 | ; | |
2479 | @end group | |
2480 | @end example | |
2481 | ||
2482 | @noindent | |
2483 | Now Bison can execute the action in the rule for @code{subroutine} without | |
2484 | deciding which rule for @code{compound} it will eventually use. Note that | |
2485 | the action is now at the end of its rule. Any mid-rule action can be | |
2486 | converted to an end-of-rule action in this way, and this is what Bison | |
2487 | actually does to implement mid-rule actions. | |
2488 | ||
847bf1f5 AD |
2489 | @node Locations, Declarations, Semantics, Grammar File |
2490 | @section Tracking Locations | |
2491 | @cindex location | |
2492 | @cindex textual position | |
2493 | @cindex position, textual | |
2494 | ||
2495 | Though grammar rules and semantic actions are enough to write a fully | |
2496 | functional parser, it can be useful to process some additionnal informations, | |
3e259915 MA |
2497 | especially symbol locations. |
2498 | ||
2499 | @c (terminal or not) ? | |
847bf1f5 AD |
2500 | |
2501 | The way locations are handled is defined by providing a data type, and actions | |
2502 | to take when rules are matched. | |
2503 | ||
2504 | @menu | |
2505 | * Location Type:: Specifying a data type for locations. | |
2506 | * Actions and Locations:: Using locations in actions. | |
2507 | * Location Default Action:: Defining a general way to compute locations. | |
2508 | @end menu | |
2509 | ||
2510 | @node Location Type, Actions and Locations, , Locations | |
2511 | @subsection Data Type of Locations | |
2512 | @cindex data type of locations | |
2513 | @cindex default location type | |
2514 | ||
2515 | Defining a data type for locations is much simpler than for semantic values, | |
2516 | since all tokens and groupings always use the same type. | |
2517 | ||
2518 | The type of locations is specified by defining a macro called @code{YYLTYPE}. | |
2519 | When @code{YYLTYPE} is not defined, Bison uses a default structure type with | |
2520 | four members: | |
2521 | ||
2522 | @example | |
2523 | struct | |
2524 | @{ | |
2525 | int first_line; | |
2526 | int first_column; | |
2527 | int last_line; | |
2528 | int last_column; | |
2529 | @} | |
2530 | @end example | |
2531 | ||
2532 | @node Actions and Locations, Location Default Action, Location Type, Locations | |
2533 | @subsection Actions and Locations | |
2534 | @cindex location actions | |
2535 | @cindex actions, location | |
2536 | @vindex @@$ | |
2537 | @vindex @@@var{n} | |
2538 | ||
2539 | Actions are not only useful for defining language semantics, but also for | |
2540 | describing the behavior of the output parser with locations. | |
2541 | ||
2542 | The most obvious way for building locations of syntactic groupings is very | |
2543 | similar to the way semantic values are computed. In a given rule, several | |
2544 | constructs can be used to access the locations of the elements being matched. | |
2545 | The location of the @var{n}th component of the right hand side is | |
2546 | @code{@@@var{n}}, while the location of the left hand side grouping is | |
2547 | @code{@@$}. | |
2548 | ||
3e259915 | 2549 | Here is a basic example using the default data type for locations: |
847bf1f5 AD |
2550 | |
2551 | @example | |
2552 | @group | |
2553 | exp: @dots{} | |
3e259915 | 2554 | | exp '/' exp |
847bf1f5 | 2555 | @{ |
3e259915 MA |
2556 | @@$.first_column = @@1.first_column; |
2557 | @@$.first_line = @@1.first_line; | |
847bf1f5 AD |
2558 | @@$.last_column = @@3.last_column; |
2559 | @@$.last_line = @@3.last_line; | |
3e259915 MA |
2560 | if ($3) |
2561 | $$ = $1 / $3; | |
2562 | else | |
2563 | @{ | |
2564 | $$ = 1; | |
2565 | printf("Division by zero, l%d,c%d-l%d,c%d", | |
2566 | @@3.first_line, @@3.first_column, | |
2567 | @@3.last_line, @@3.last_column); | |
2568 | @} | |
847bf1f5 AD |
2569 | @} |
2570 | @end group | |
2571 | @end example | |
2572 | ||
3e259915 MA |
2573 | As for semantic values, there is a default action for locations that is |
2574 | run each time a rule is matched. It sets the beginning of @code{@@$} to the | |
2575 | beginning of the first symbol, and the end of @code{@@$} to the end of the | |
79282c6c | 2576 | last symbol. |
3e259915 MA |
2577 | |
2578 | With this default action, the location tracking can be fully automatic. The | |
2579 | example above simply rewrites this way: | |
2580 | ||
2581 | @example | |
2582 | @group | |
2583 | exp: @dots{} | |
2584 | | exp '/' exp | |
2585 | @{ | |
2586 | if ($3) | |
2587 | $$ = $1 / $3; | |
2588 | else | |
2589 | @{ | |
2590 | $$ = 1; | |
2591 | printf("Division by zero, l%d,c%d-l%d,c%d", | |
2592 | @@3.first_line, @@3.first_column, | |
2593 | @@3.last_line, @@3.last_column); | |
2594 | @} | |
2595 | @} | |
2596 | @end group | |
2597 | @end example | |
847bf1f5 AD |
2598 | |
2599 | @node Location Default Action, , Actions and Locations, Locations | |
2600 | @subsection Default Action for Locations | |
2601 | @vindex YYLLOC_DEFAULT | |
2602 | ||
2603 | Actually, actions are not the best place to compute locations. Since locations | |
2604 | are much more general than semantic values, there is room in the output parser | |
79282c6c | 2605 | to redefine the default action to take for each rule. The |
3e259915 | 2606 | @code{YYLLOC_DEFAULT} macro is called each time a rule is matched, before the |
79282c6c | 2607 | associated action is run. |
847bf1f5 | 2608 | |
3e259915 | 2609 | Most of the time, this macro is general enough to suppress location |
79282c6c | 2610 | dedicated code from semantic actions. |
847bf1f5 | 2611 | |
79282c6c AD |
2612 | The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is |
2613 | the location of the grouping (the result of the computation). The second one | |
2614 | is an array holding locations of all right hand side elements of the rule | |
3e259915 | 2615 | being matched. The last one is the size of the right hand side rule. |
847bf1f5 | 2616 | |
3e259915 | 2617 | By default, it is defined this way: |
847bf1f5 AD |
2618 | |
2619 | @example | |
2620 | @group | |
3e259915 MA |
2621 | #define YYLLOC_DEFAULT(Current, Rhs, N) \ |
2622 | Current.last_line = Rhs[N].last_line; \ | |
2623 | Current.last_column = Rhs[N].last_column; | |
847bf1f5 AD |
2624 | @end group |
2625 | @end example | |
2626 | ||
3e259915 | 2627 | When defining @code{YYLLOC_DEFAULT}, you should consider that: |
847bf1f5 | 2628 | |
3e259915 | 2629 | @itemize @bullet |
79282c6c | 2630 | @item |
3e259915 MA |
2631 | All arguments are free of side-effects. However, only the first one (the |
2632 | result) should be modified by @code{YYLLOC_DEFAULT}. | |
847bf1f5 | 2633 | |
3e259915 MA |
2634 | @item |
2635 | Before @code{YYLLOC_DEFAULT} is executed, the output parser sets @code{@@$} | |
2636 | to @code{@@1}. | |
2637 | ||
2638 | @item | |
2639 | For consistency with semantic actions, valid indexes for the location array | |
2640 | range from 1 to @var{n}. | |
2641 | @end itemize | |
847bf1f5 AD |
2642 | |
2643 | @node Declarations, Multiple Parsers, Locations, Grammar File | |
bfa74976 RS |
2644 | @section Bison Declarations |
2645 | @cindex declarations, Bison | |
2646 | @cindex Bison declarations | |
2647 | ||
2648 | The @dfn{Bison declarations} section of a Bison grammar defines the symbols | |
2649 | used in formulating the grammar and the data types of semantic values. | |
2650 | @xref{Symbols}. | |
2651 | ||
2652 | All token type names (but not single-character literal tokens such as | |
2653 | @code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be | |
2654 | declared if you need to specify which data type to use for the semantic | |
2655 | value (@pxref{Multiple Types, ,More Than One Value Type}). | |
2656 | ||
2657 | The first rule in the file also specifies the start symbol, by default. | |
2658 | If you want some other symbol to be the start symbol, you must declare | |
2659 | it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free Grammars}). | |
2660 | ||
2661 | @menu | |
2662 | * Token Decl:: Declaring terminal symbols. | |
2663 | * Precedence Decl:: Declaring terminals with precedence and associativity. | |
2664 | * Union Decl:: Declaring the set of all semantic value types. | |
2665 | * Type Decl:: Declaring the choice of type for a nonterminal symbol. | |
2666 | * Expect Decl:: Suppressing warnings about shift/reduce conflicts. | |
2667 | * Start Decl:: Specifying the start symbol. | |
2668 | * Pure Decl:: Requesting a reentrant parser. | |
2669 | * Decl Summary:: Table of all Bison declarations. | |
2670 | @end menu | |
2671 | ||
2672 | @node Token Decl, Precedence Decl, , Declarations | |
2673 | @subsection Token Type Names | |
2674 | @cindex declaring token type names | |
2675 | @cindex token type names, declaring | |
931c7513 | 2676 | @cindex declaring literal string tokens |
bfa74976 RS |
2677 | @findex %token |
2678 | ||
2679 | The basic way to declare a token type name (terminal symbol) is as follows: | |
2680 | ||
2681 | @example | |
2682 | %token @var{name} | |
2683 | @end example | |
2684 | ||
2685 | Bison will convert this into a @code{#define} directive in | |
2686 | the parser, so that the function @code{yylex} (if it is in this file) | |
2687 | can use the name @var{name} to stand for this token type's code. | |
2688 | ||
14ded682 AD |
2689 | Alternatively, you can use @code{%left}, @code{%right}, or |
2690 | @code{%nonassoc} instead of @code{%token}, if you wish to specify | |
2691 | associativity and precedence. @xref{Precedence Decl, ,Operator | |
2692 | Precedence}. | |
bfa74976 RS |
2693 | |
2694 | You can explicitly specify the numeric code for a token type by appending | |
2695 | an integer value in the field immediately following the token name: | |
2696 | ||
2697 | @example | |
2698 | %token NUM 300 | |
2699 | @end example | |
2700 | ||
2701 | @noindent | |
2702 | It is generally best, however, to let Bison choose the numeric codes for | |
2703 | all token types. Bison will automatically select codes that don't conflict | |
2704 | with each other or with ASCII characters. | |
2705 | ||
2706 | In the event that the stack type is a union, you must augment the | |
2707 | @code{%token} or other token declaration to include the data type | |
13863333 | 2708 | alternative delimited by angle-brackets (@pxref{Multiple Types, ,More Than One Value Type}). |
bfa74976 RS |
2709 | |
2710 | For example: | |
2711 | ||
2712 | @example | |
2713 | @group | |
2714 | %union @{ /* define stack type */ | |
2715 | double val; | |
2716 | symrec *tptr; | |
2717 | @} | |
2718 | %token <val> NUM /* define token NUM and its type */ | |
2719 | @end group | |
2720 | @end example | |
2721 | ||
931c7513 RS |
2722 | You can associate a literal string token with a token type name by |
2723 | writing the literal string at the end of a @code{%token} | |
2724 | declaration which declares the name. For example: | |
2725 | ||
2726 | @example | |
2727 | %token arrow "=>" | |
2728 | @end example | |
2729 | ||
2730 | @noindent | |
2731 | For example, a grammar for the C language might specify these names with | |
2732 | equivalent literal string tokens: | |
2733 | ||
2734 | @example | |
2735 | %token <operator> OR "||" | |
2736 | %token <operator> LE 134 "<=" | |
2737 | %left OR "<=" | |
2738 | @end example | |
2739 | ||
2740 | @noindent | |
2741 | Once you equate the literal string and the token name, you can use them | |
2742 | interchangeably in further declarations or the grammar rules. The | |
2743 | @code{yylex} function can use the token name or the literal string to | |
2744 | obtain the token type code number (@pxref{Calling Convention}). | |
2745 | ||
bfa74976 RS |
2746 | @node Precedence Decl, Union Decl, Token Decl, Declarations |
2747 | @subsection Operator Precedence | |
2748 | @cindex precedence declarations | |
2749 | @cindex declaring operator precedence | |
2750 | @cindex operator precedence, declaring | |
2751 | ||
2752 | Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to | |
2753 | declare a token and specify its precedence and associativity, all at | |
2754 | once. These are called @dfn{precedence declarations}. | |
2755 | @xref{Precedence, ,Operator Precedence}, for general information on operator precedence. | |
2756 | ||
2757 | The syntax of a precedence declaration is the same as that of | |
2758 | @code{%token}: either | |
2759 | ||
2760 | @example | |
2761 | %left @var{symbols}@dots{} | |
2762 | @end example | |
2763 | ||
2764 | @noindent | |
2765 | or | |
2766 | ||
2767 | @example | |
2768 | %left <@var{type}> @var{symbols}@dots{} | |
2769 | @end example | |
2770 | ||
2771 | And indeed any of these declarations serves the purposes of @code{%token}. | |
2772 | But in addition, they specify the associativity and relative precedence for | |
2773 | all the @var{symbols}: | |
2774 | ||
2775 | @itemize @bullet | |
2776 | @item | |
2777 | The associativity of an operator @var{op} determines how repeated uses | |
2778 | of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op} | |
2779 | @var{z}} is parsed by grouping @var{x} with @var{y} first or by | |
2780 | grouping @var{y} with @var{z} first. @code{%left} specifies | |
2781 | left-associativity (grouping @var{x} with @var{y} first) and | |
2782 | @code{%right} specifies right-associativity (grouping @var{y} with | |
2783 | @var{z} first). @code{%nonassoc} specifies no associativity, which | |
2784 | means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is | |
2785 | considered a syntax error. | |
2786 | ||
2787 | @item | |
2788 | The precedence of an operator determines how it nests with other operators. | |
2789 | All the tokens declared in a single precedence declaration have equal | |
2790 | precedence and nest together according to their associativity. | |
2791 | When two tokens declared in different precedence declarations associate, | |
2792 | the one declared later has the higher precedence and is grouped first. | |
2793 | @end itemize | |
2794 | ||
2795 | @node Union Decl, Type Decl, Precedence Decl, Declarations | |
2796 | @subsection The Collection of Value Types | |
2797 | @cindex declaring value types | |
2798 | @cindex value types, declaring | |
2799 | @findex %union | |
2800 | ||
2801 | The @code{%union} declaration specifies the entire collection of possible | |
2802 | data types for semantic values. The keyword @code{%union} is followed by a | |
2803 | pair of braces containing the same thing that goes inside a @code{union} in | |
13863333 | 2804 | C. |
bfa74976 RS |
2805 | |
2806 | For example: | |
2807 | ||
2808 | @example | |
2809 | @group | |
2810 | %union @{ | |
2811 | double val; | |
2812 | symrec *tptr; | |
2813 | @} | |
2814 | @end group | |
2815 | @end example | |
2816 | ||
2817 | @noindent | |
2818 | This says that the two alternative types are @code{double} and @code{symrec | |
2819 | *}. They are given names @code{val} and @code{tptr}; these names are used | |
2820 | in the @code{%token} and @code{%type} declarations to pick one of the types | |
2821 | for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}). | |
2822 | ||
2823 | Note that, unlike making a @code{union} declaration in C, you do not write | |
2824 | a semicolon after the closing brace. | |
2825 | ||
2826 | @node Type Decl, Expect Decl, Union Decl, Declarations | |
2827 | @subsection Nonterminal Symbols | |
2828 | @cindex declaring value types, nonterminals | |
2829 | @cindex value types, nonterminals, declaring | |
2830 | @findex %type | |
2831 | ||
2832 | @noindent | |
2833 | When you use @code{%union} to specify multiple value types, you must | |
2834 | declare the value type of each nonterminal symbol for which values are | |
2835 | used. This is done with a @code{%type} declaration, like this: | |
2836 | ||
2837 | @example | |
2838 | %type <@var{type}> @var{nonterminal}@dots{} | |
2839 | @end example | |
2840 | ||
2841 | @noindent | |
2842 | Here @var{nonterminal} is the name of a nonterminal symbol, and @var{type} | |
2843 | is the name given in the @code{%union} to the alternative that you want | |
2844 | (@pxref{Union Decl, ,The Collection of Value Types}). You can give any number of nonterminal symbols in | |
2845 | the same @code{%type} declaration, if they have the same value type. Use | |
2846 | spaces to separate the symbol names. | |
2847 | ||
931c7513 RS |
2848 | You can also declare the value type of a terminal symbol. To do this, |
2849 | use the same @code{<@var{type}>} construction in a declaration for the | |
2850 | terminal symbol. All kinds of token declarations allow | |
2851 | @code{<@var{type}>}. | |
2852 | ||
bfa74976 RS |
2853 | @node Expect Decl, Start Decl, Type Decl, Declarations |
2854 | @subsection Suppressing Conflict Warnings | |
2855 | @cindex suppressing conflict warnings | |
2856 | @cindex preventing warnings about conflicts | |
2857 | @cindex warnings, preventing | |
2858 | @cindex conflicts, suppressing warnings of | |
2859 | @findex %expect | |
2860 | ||
2861 | Bison normally warns if there are any conflicts in the grammar | |
2862 | (@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars have harmless shift/reduce | |
2863 | conflicts which are resolved in a predictable way and would be difficult to | |
2864 | eliminate. It is desirable to suppress the warning about these conflicts | |
2865 | unless the number of conflicts changes. You can do this with the | |
2866 | @code{%expect} declaration. | |
2867 | ||
2868 | The declaration looks like this: | |
2869 | ||
2870 | @example | |
2871 | %expect @var{n} | |
2872 | @end example | |
2873 | ||
2874 | Here @var{n} is a decimal integer. The declaration says there should be no | |
2875 | warning if there are @var{n} shift/reduce conflicts and no reduce/reduce | |
2876 | conflicts. The usual warning is given if there are either more or fewer | |
2877 | conflicts, or if there are any reduce/reduce conflicts. | |
2878 | ||
2879 | In general, using @code{%expect} involves these steps: | |
2880 | ||
2881 | @itemize @bullet | |
2882 | @item | |
2883 | Compile your grammar without @code{%expect}. Use the @samp{-v} option | |
2884 | to get a verbose list of where the conflicts occur. Bison will also | |
2885 | print the number of conflicts. | |
2886 | ||
2887 | @item | |
2888 | Check each of the conflicts to make sure that Bison's default | |
2889 | resolution is what you really want. If not, rewrite the grammar and | |
2890 | go back to the beginning. | |
2891 | ||
2892 | @item | |
2893 | Add an @code{%expect} declaration, copying the number @var{n} from the | |
2894 | number which Bison printed. | |
2895 | @end itemize | |
2896 | ||
2897 | Now Bison will stop annoying you about the conflicts you have checked, but | |
2898 | it will warn you again if changes in the grammar result in additional | |
2899 | conflicts. | |
2900 | ||
2901 | @node Start Decl, Pure Decl, Expect Decl, Declarations | |
2902 | @subsection The Start-Symbol | |
2903 | @cindex declaring the start symbol | |
2904 | @cindex start symbol, declaring | |
2905 | @cindex default start symbol | |
2906 | @findex %start | |
2907 | ||
2908 | Bison assumes by default that the start symbol for the grammar is the first | |
2909 | nonterminal specified in the grammar specification section. The programmer | |
2910 | may override this restriction with the @code{%start} declaration as follows: | |
2911 | ||
2912 | @example | |
2913 | %start @var{symbol} | |
2914 | @end example | |
2915 | ||
2916 | @node Pure Decl, Decl Summary, Start Decl, Declarations | |
2917 | @subsection A Pure (Reentrant) Parser | |
2918 | @cindex reentrant parser | |
2919 | @cindex pure parser | |
2920 | @findex %pure_parser | |
2921 | ||
2922 | A @dfn{reentrant} program is one which does not alter in the course of | |
2923 | execution; in other words, it consists entirely of @dfn{pure} (read-only) | |
2924 | code. Reentrancy is important whenever asynchronous execution is possible; | |
14ded682 AD |
2925 | for example, a non-reentrant program may not be safe to call from a signal |
2926 | handler. In systems with multiple threads of control, a non-reentrant | |
bfa74976 RS |
2927 | program must be called only within interlocks. |
2928 | ||
70811b85 RS |
2929 | Normally, Bison generates a parser which is not reentrant. This is |
2930 | suitable for most uses, and it permits compatibility with YACC. (The | |
2931 | standard YACC interfaces are inherently nonreentrant, because they use | |
2932 | statically allocated variables for communication with @code{yylex}, | |
2933 | including @code{yylval} and @code{yylloc}.) | |
bfa74976 | 2934 | |
70811b85 RS |
2935 | Alternatively, you can generate a pure, reentrant parser. The Bison |
2936 | declaration @code{%pure_parser} says that you want the parser to be | |
2937 | reentrant. It looks like this: | |
bfa74976 RS |
2938 | |
2939 | @example | |
2940 | %pure_parser | |
2941 | @end example | |
2942 | ||
70811b85 RS |
2943 | The result is that the communication variables @code{yylval} and |
2944 | @code{yylloc} become local variables in @code{yyparse}, and a different | |
2945 | calling convention is used for the lexical analyzer function | |
2946 | @code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure | |
2947 | Parsers}, for the details of this. The variable @code{yynerrs} also | |
2948 | becomes local in @code{yyparse} (@pxref{Error Reporting, ,The Error | |
2949 | Reporting Function @code{yyerror}}). The convention for calling | |
2950 | @code{yyparse} itself is unchanged. | |
2951 | ||
2952 | Whether the parser is pure has nothing to do with the grammar rules. | |
2953 | You can generate either a pure parser or a nonreentrant parser from any | |
2954 | valid grammar. | |
bfa74976 RS |
2955 | |
2956 | @node Decl Summary, , Pure Decl, Declarations | |
2957 | @subsection Bison Declaration Summary | |
2958 | @cindex Bison declaration summary | |
2959 | @cindex declaration summary | |
2960 | @cindex summary, Bison declaration | |
2961 | ||
2962 | Here is a summary of all Bison declarations: | |
2963 | ||
2964 | @table @code | |
2965 | @item %union | |
2966 | Declare the collection of data types that semantic values may have | |
2967 | (@pxref{Union Decl, ,The Collection of Value Types}). | |
2968 | ||
2969 | @item %token | |
2970 | Declare a terminal symbol (token type name) with no precedence | |
2971 | or associativity specified (@pxref{Token Decl, ,Token Type Names}). | |
2972 | ||
2973 | @item %right | |
2974 | Declare a terminal symbol (token type name) that is right-associative | |
2975 | (@pxref{Precedence Decl, ,Operator Precedence}). | |
2976 | ||
2977 | @item %left | |
2978 | Declare a terminal symbol (token type name) that is left-associative | |
2979 | (@pxref{Precedence Decl, ,Operator Precedence}). | |
2980 | ||
2981 | @item %nonassoc | |
2982 | Declare a terminal symbol (token type name) that is nonassociative | |
2983 | (using it in a way that would be associative is a syntax error) | |
2984 | (@pxref{Precedence Decl, ,Operator Precedence}). | |
2985 | ||
2986 | @item %type | |
2987 | Declare the type of semantic values for a nonterminal symbol | |
2988 | (@pxref{Type Decl, ,Nonterminal Symbols}). | |
2989 | ||
2990 | @item %start | |
89cab50d AD |
2991 | Specify the grammar's start symbol (@pxref{Start Decl, ,The |
2992 | Start-Symbol}). | |
bfa74976 RS |
2993 | |
2994 | @item %expect | |
2995 | Declare the expected number of shift-reduce conflicts | |
2996 | (@pxref{Expect Decl, ,Suppressing Conflict Warnings}). | |
2997 | ||
6deb4447 AD |
2998 | @item %yacc |
2999 | @itemx %fixed_output_files | |
3000 | Pretend the option @option{--yacc} was given, i.e., imitate Yacc, | |
3001 | including its naming conventions. @xref{Bison Options}, for more. | |
3002 | ||
89cab50d AD |
3003 | @item %locations |
3004 | Generate the code processing the locations (@pxref{Action Features, | |
3005 | ,Special Features for Use in Actions}). This mode is enabled as soon as | |
3006 | the grammar uses the special @samp{@@@var{n}} tokens, but if your | |
3007 | grammar does not use it, using @samp{%locations} allows for more | |
3008 | accurate parse error messages. | |
3009 | ||
bfa74976 | 3010 | @item %pure_parser |
89cab50d AD |
3011 | Request a pure (reentrant) parser program (@pxref{Pure Decl, ,A Pure |
3012 | (Reentrant) Parser}). | |
931c7513 | 3013 | |
6deb4447 AD |
3014 | @item %no_parser |
3015 | Do not include any C code in the parser file; generate tables only. The | |
3016 | parser file contains just @code{#define} directives and static variable | |
3017 | declarations. | |
3018 | ||
3019 | This option also tells Bison to write the C code for the grammar actions | |
3020 | into a file named @file{@var{filename}.act}, in the form of a | |
3021 | brace-surrounded body fit for a @code{switch} statement. | |
3022 | ||
931c7513 RS |
3023 | @item %no_lines |
3024 | Don't generate any @code{#line} preprocessor commands in the parser | |
3025 | file. Ordinarily Bison writes these commands in the parser file so that | |
3026 | the C compiler and debuggers will associate errors and object code with | |
3027 | your source file (the grammar file). This directive causes them to | |
3028 | associate errors with the parser file, treating it an independent source | |
3029 | file in its own right. | |
3030 | ||
6deb4447 AD |
3031 | @item %debug |
3032 | Output a definition of the macro @code{YYDEBUG} into the parser file, so | |
3033 | that the debugging facilities are compiled. @xref{Debugging, ,Debugging | |
3034 | Your Parser}. | |
3035 | ||
3036 | @item %defines | |
3037 | Write an extra output file containing macro definitions for the token | |
3038 | type names defined in the grammar and the semantic value type | |
3039 | @code{YYSTYPE}, as well as a few @code{extern} variable declarations. | |
3040 | ||
3041 | If the parser output file is named @file{@var{name}.c} then this file | |
3042 | is named @file{@var{name}.h}.@refill | |
3043 | ||
3044 | This output file is essential if you wish to put the definition of | |
3045 | @code{yylex} in a separate source file, because @code{yylex} needs to | |
3046 | be able to refer to token type codes and the variable | |
3047 | @code{yylval}. @xref{Token Values, ,Semantic Values of Tokens}.@refill | |
3048 | ||
f9a8293a AD |
3049 | @c @item %source_extension |
3050 | @c Specify the extension of the parser output file. | |
3051 | @c | |
3052 | @c For example, a grammar file named @file{foo.yy} and containing a | |
3053 | @c @code{%source_extension .cpp} directive will produce a parser file | |
3054 | @c named @file{foo.tab.cpp} | |
3055 | @c | |
3056 | @c @item %header_extension | |
3057 | @c Specify the extension of the parser header file generated when | |
3058 | @c @code{%define} or @samp{-d} are used. | |
3059 | @c | |
3060 | @c For example, a garmmar file named @file{foo.ypp} and containing a | |
3061 | @c @code{%header_extension .hh} directive will produce a header file | |
3062 | @c named @file{foo.tab.hh} | |
3063 | ||
6deb4447 AD |
3064 | @item %verbose |
3065 | Write an extra output file containing verbose descriptions of the | |
3066 | parser states and what is done for each type of look-ahead token in | |
3067 | that state. | |
3068 | ||
3069 | This file also describes all the conflicts, both those resolved by | |
3070 | operator precedence and the unresolved ones. | |
3071 | ||
3072 | The file's name is made by removing @samp{.tab.c} or @samp{.c} from | |
3073 | the parser output file name, and adding @samp{.output} instead.@refill | |
3074 | ||
3075 | Therefore, if the input file is @file{foo.y}, then the parser file is | |
3076 | called @file{foo.tab.c} by default. As a consequence, the verbose | |
3077 | output file is called @file{foo.output}.@refill | |
3078 | ||
931c7513 RS |
3079 | @item %token_table |
3080 | Generate an array of token names in the parser file. The name of the | |
3081 | array is @code{yytname}; @code{yytname[@var{i}]} is the name of the | |
3082 | token whose internal Bison token code number is @var{i}. The first three | |
3083 | elements of @code{yytname} are always @code{"$"}, @code{"error"}, and | |
3084 | @code{"$illegal"}; after these come the symbols defined in the grammar | |
3085 | file. | |
3086 | ||
3087 | For single-character literal tokens and literal string tokens, the name | |
3088 | in the table includes the single-quote or double-quote characters: for | |
3089 | example, @code{"'+'"} is a single-character literal and @code{"\"<=\""} | |
3090 | is a literal string token. All the characters of the literal string | |
3091 | token appear verbatim in the string found in the table; even | |
3092 | double-quote characters are not escaped. For example, if the token | |
3093 | consists of three characters @samp{*"*}, its string in @code{yytname} | |
3094 | contains @samp{"*"*"}. (In C, that would be written as | |
3095 | @code{"\"*\"*\""}). | |
3096 | ||
3097 | When you specify @code{%token_table}, Bison also generates macro | |
3098 | definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and | |
3099 | @code{YYNRULES}, and @code{YYNSTATES}: | |
3100 | ||
3101 | @table @code | |
3102 | @item YYNTOKENS | |
3103 | The highest token number, plus one. | |
3104 | @item YYNNTS | |
9ecbd125 | 3105 | The number of nonterminal symbols. |
931c7513 RS |
3106 | @item YYNRULES |
3107 | The number of grammar rules, | |
3108 | @item YYNSTATES | |
3109 | The number of parser states (@pxref{Parser States}). | |
3110 | @end table | |
bfa74976 RS |
3111 | @end table |
3112 | ||
931c7513 | 3113 | @node Multiple Parsers,, Declarations, Grammar File |
bfa74976 RS |
3114 | @section Multiple Parsers in the Same Program |
3115 | ||
3116 | Most programs that use Bison parse only one language and therefore contain | |
3117 | only one Bison parser. But what if you want to parse more than one | |
3118 | language with the same program? Then you need to avoid a name conflict | |
3119 | between different definitions of @code{yyparse}, @code{yylval}, and so on. | |
3120 | ||
3121 | The easy way to do this is to use the option @samp{-p @var{prefix}} | |
3122 | (@pxref{Invocation, ,Invoking Bison}). This renames the interface functions and | |
3123 | variables of the Bison parser to start with @var{prefix} instead of | |
3124 | @samp{yy}. You can use this to give each parser distinct names that do | |
3125 | not conflict. | |
3126 | ||
3127 | The precise list of symbols renamed is @code{yyparse}, @code{yylex}, | |
c656404a RS |
3128 | @code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yychar} and |
3129 | @code{yydebug}. For example, if you use @samp{-p c}, the names become | |
3130 | @code{cparse}, @code{clex}, and so on. | |
bfa74976 RS |
3131 | |
3132 | @strong{All the other variables and macros associated with Bison are not | |
3133 | renamed.} These others are not global; there is no conflict if the same | |
3134 | name is used in different parsers. For example, @code{YYSTYPE} is not | |
3135 | renamed, but defining this in different ways in different parsers causes | |
3136 | no trouble (@pxref{Value Type, ,Data Types of Semantic Values}). | |
3137 | ||
3138 | The @samp{-p} option works by adding macro definitions to the beginning | |
3139 | of the parser source file, defining @code{yyparse} as | |
3140 | @code{@var{prefix}parse}, and so on. This effectively substitutes one | |
3141 | name for the other in the entire parser file. | |
3142 | ||
3143 | @node Interface, Algorithm, Grammar File, Top | |
3144 | @chapter Parser C-Language Interface | |
3145 | @cindex C-language interface | |
3146 | @cindex interface | |
3147 | ||
3148 | The Bison parser is actually a C function named @code{yyparse}. Here we | |
3149 | describe the interface conventions of @code{yyparse} and the other | |
3150 | functions that it needs to use. | |
3151 | ||
3152 | Keep in mind that the parser uses many C identifiers starting with | |
3153 | @samp{yy} and @samp{YY} for internal purposes. If you use such an | |
75f5aaea MA |
3154 | identifier (aside from those in this manual) in an action or in epilogue |
3155 | in the grammar file, you are likely to run into trouble. | |
bfa74976 RS |
3156 | |
3157 | @menu | |
3158 | * Parser Function:: How to call @code{yyparse} and what it returns. | |
13863333 | 3159 | * Lexical:: You must supply a function @code{yylex} |
bfa74976 RS |
3160 | which reads tokens. |
3161 | * Error Reporting:: You must supply a function @code{yyerror}. | |
3162 | * Action Features:: Special features for use in actions. | |
3163 | @end menu | |
3164 | ||
3165 | @node Parser Function, Lexical, , Interface | |
3166 | @section The Parser Function @code{yyparse} | |
3167 | @findex yyparse | |
3168 | ||
3169 | You call the function @code{yyparse} to cause parsing to occur. This | |
3170 | function reads tokens, executes actions, and ultimately returns when it | |
3171 | encounters end-of-input or an unrecoverable syntax error. You can also | |
14ded682 AD |
3172 | write an action which directs @code{yyparse} to return immediately |
3173 | without reading further. | |
bfa74976 RS |
3174 | |
3175 | The value returned by @code{yyparse} is 0 if parsing was successful (return | |
3176 | is due to end-of-input). | |
3177 | ||
3178 | The value is 1 if parsing failed (return is due to a syntax error). | |
3179 | ||
3180 | In an action, you can cause immediate return from @code{yyparse} by using | |
3181 | these macros: | |
3182 | ||
3183 | @table @code | |
3184 | @item YYACCEPT | |
3185 | @findex YYACCEPT | |
3186 | Return immediately with value 0 (to report success). | |
3187 | ||
3188 | @item YYABORT | |
3189 | @findex YYABORT | |
3190 | Return immediately with value 1 (to report failure). | |
3191 | @end table | |
3192 | ||
3193 | @node Lexical, Error Reporting, Parser Function, Interface | |
3194 | @section The Lexical Analyzer Function @code{yylex} | |
3195 | @findex yylex | |
3196 | @cindex lexical analyzer | |
3197 | ||
3198 | The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from | |
3199 | the input stream and returns them to the parser. Bison does not create | |
3200 | this function automatically; you must write it so that @code{yyparse} can | |
3201 | call it. The function is sometimes referred to as a lexical scanner. | |
3202 | ||
3203 | In simple programs, @code{yylex} is often defined at the end of the Bison | |
3204 | grammar file. If @code{yylex} is defined in a separate source file, you | |
3205 | need to arrange for the token-type macro definitions to be available there. | |
3206 | To do this, use the @samp{-d} option when you run Bison, so that it will | |
3207 | write these macro definitions into a separate header file | |
3208 | @file{@var{name}.tab.h} which you can include in the other source files | |
3209 | that need it. @xref{Invocation, ,Invoking Bison}.@refill | |
3210 | ||
3211 | @menu | |
3212 | * Calling Convention:: How @code{yyparse} calls @code{yylex}. | |
3213 | * Token Values:: How @code{yylex} must return the semantic value | |
3214 | of the token it has read. | |
3215 | * Token Positions:: How @code{yylex} must return the text position | |
3216 | (line number, etc.) of the token, if the | |
3217 | actions want that. | |
3218 | * Pure Calling:: How the calling convention differs | |
3219 | in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). | |
3220 | @end menu | |
3221 | ||
3222 | @node Calling Convention, Token Values, , Lexical | |
3223 | @subsection Calling Convention for @code{yylex} | |
3224 | ||
3225 | The value that @code{yylex} returns must be the numeric code for the type | |
3226 | of token it has just found, or 0 for end-of-input. | |
3227 | ||
3228 | When a token is referred to in the grammar rules by a name, that name | |
3229 | in the parser file becomes a C macro whose definition is the proper | |
3230 | numeric code for that token type. So @code{yylex} can use the name | |
3231 | to indicate that type. @xref{Symbols}. | |
3232 | ||
3233 | When a token is referred to in the grammar rules by a character literal, | |
3234 | the numeric code for that character is also the code for the token type. | |
3235 | So @code{yylex} can simply return that character code. The null character | |
3236 | must not be used this way, because its code is zero and that is what | |
3237 | signifies end-of-input. | |
3238 | ||
3239 | Here is an example showing these things: | |
3240 | ||
3241 | @example | |
13863333 AD |
3242 | int |
3243 | yylex (void) | |
bfa74976 RS |
3244 | @{ |
3245 | @dots{} | |
3246 | if (c == EOF) /* Detect end of file. */ | |
3247 | return 0; | |
3248 | @dots{} | |
3249 | if (c == '+' || c == '-') | |
3250 | return c; /* Assume token type for `+' is '+'. */ | |
3251 | @dots{} | |
3252 | return INT; /* Return the type of the token. */ | |
3253 | @dots{} | |
3254 | @} | |
3255 | @end example | |
3256 | ||
3257 | @noindent | |
3258 | This interface has been designed so that the output from the @code{lex} | |
3259 | utility can be used without change as the definition of @code{yylex}. | |
3260 | ||
931c7513 RS |
3261 | If the grammar uses literal string tokens, there are two ways that |
3262 | @code{yylex} can determine the token type codes for them: | |
3263 | ||
3264 | @itemize @bullet | |
3265 | @item | |
3266 | If the grammar defines symbolic token names as aliases for the | |
3267 | literal string tokens, @code{yylex} can use these symbolic names like | |
3268 | all others. In this case, the use of the literal string tokens in | |
3269 | the grammar file has no effect on @code{yylex}. | |
3270 | ||
3271 | @item | |
9ecbd125 | 3272 | @code{yylex} can find the multicharacter token in the @code{yytname} |
931c7513 | 3273 | table. The index of the token in the table is the token type's code. |
9ecbd125 | 3274 | The name of a multicharacter token is recorded in @code{yytname} with a |
931c7513 RS |
3275 | double-quote, the token's characters, and another double-quote. The |
3276 | token's characters are not escaped in any way; they appear verbatim in | |
3277 | the contents of the string in the table. | |
3278 | ||
3279 | Here's code for looking up a token in @code{yytname}, assuming that the | |
3280 | characters of the token are stored in @code{token_buffer}. | |
3281 | ||
3282 | @smallexample | |
3283 | for (i = 0; i < YYNTOKENS; i++) | |
3284 | @{ | |
3285 | if (yytname[i] != 0 | |
3286 | && yytname[i][0] == '"' | |
6f515a27 JT |
3287 | && strncmp (yytname[i] + 1, token_buffer, |
3288 | strlen (token_buffer)) | |
931c7513 RS |
3289 | && yytname[i][strlen (token_buffer) + 1] == '"' |
3290 | && yytname[i][strlen (token_buffer) + 2] == 0) | |
3291 | break; | |
3292 | @} | |
3293 | @end smallexample | |
3294 | ||
3295 | The @code{yytname} table is generated only if you use the | |
3296 | @code{%token_table} declaration. @xref{Decl Summary}. | |
3297 | @end itemize | |
3298 | ||
bfa74976 RS |
3299 | @node Token Values, Token Positions, Calling Convention, Lexical |
3300 | @subsection Semantic Values of Tokens | |
3301 | ||
3302 | @vindex yylval | |
14ded682 | 3303 | In an ordinary (non-reentrant) parser, the semantic value of the token must |
bfa74976 RS |
3304 | be stored into the global variable @code{yylval}. When you are using |
3305 | just one data type for semantic values, @code{yylval} has that type. | |
3306 | Thus, if the type is @code{int} (the default), you might write this in | |
3307 | @code{yylex}: | |
3308 | ||
3309 | @example | |
3310 | @group | |
3311 | @dots{} | |
3312 | yylval = value; /* Put value onto Bison stack. */ | |
3313 | return INT; /* Return the type of the token. */ | |
3314 | @dots{} | |
3315 | @end group | |
3316 | @end example | |
3317 | ||
3318 | When you are using multiple data types, @code{yylval}'s type is a union | |
3319 | made from the @code{%union} declaration (@pxref{Union Decl, ,The Collection of Value Types}). So when | |
3320 | you store a token's value, you must use the proper member of the union. | |
3321 | If the @code{%union} declaration looks like this: | |
3322 | ||
3323 | @example | |
3324 | @group | |
3325 | %union @{ | |
3326 | int intval; | |
3327 | double val; | |
3328 | symrec *tptr; | |
3329 | @} | |
3330 | @end group | |
3331 | @end example | |
3332 | ||
3333 | @noindent | |
3334 | then the code in @code{yylex} might look like this: | |
3335 | ||
3336 | @example | |
3337 | @group | |
3338 | @dots{} | |
3339 | yylval.intval = value; /* Put value onto Bison stack. */ | |
3340 | return INT; /* Return the type of the token. */ | |
3341 | @dots{} | |
3342 | @end group | |
3343 | @end example | |
3344 | ||
3345 | @node Token Positions, Pure Calling, Token Values, Lexical | |
3346 | @subsection Textual Positions of Tokens | |
3347 | ||
3348 | @vindex yylloc | |
847bf1f5 AD |
3349 | If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, , |
3350 | Tracking Locations}) in actions to keep track of the | |
89cab50d AD |
3351 | textual locations of tokens and groupings, then you must provide this |
3352 | information in @code{yylex}. The function @code{yyparse} expects to | |
3353 | find the textual location of a token just parsed in the global variable | |
3354 | @code{yylloc}. So @code{yylex} must store the proper data in that | |
847bf1f5 AD |
3355 | variable. |
3356 | ||
3357 | By default, the value of @code{yylloc} is a structure and you need only | |
89cab50d AD |
3358 | initialize the members that are going to be used by the actions. The |
3359 | four members are called @code{first_line}, @code{first_column}, | |
3360 | @code{last_line} and @code{last_column}. Note that the use of this | |
3361 | feature makes the parser noticeably slower. | |
bfa74976 RS |
3362 | |
3363 | @tindex YYLTYPE | |
3364 | The data type of @code{yylloc} has the name @code{YYLTYPE}. | |
3365 | ||
3366 | @node Pure Calling, , Token Positions, Lexical | |
c656404a | 3367 | @subsection Calling Conventions for Pure Parsers |
bfa74976 | 3368 | |
e425e872 RS |
3369 | When you use the Bison declaration @code{%pure_parser} to request a |
3370 | pure, reentrant parser, the global communication variables @code{yylval} | |
3371 | and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant) | |
3372 | Parser}.) In such parsers the two global variables are replaced by | |
3373 | pointers passed as arguments to @code{yylex}. You must declare them as | |
3374 | shown here, and pass the information back by storing it through those | |
3375 | pointers. | |
bfa74976 RS |
3376 | |
3377 | @example | |
13863333 AD |
3378 | int |
3379 | yylex (YYSTYPE *lvalp, YYLTYPE *llocp) | |
bfa74976 RS |
3380 | @{ |
3381 | @dots{} | |
3382 | *lvalp = value; /* Put value onto Bison stack. */ | |
3383 | return INT; /* Return the type of the token. */ | |
3384 | @dots{} | |
3385 | @} | |
3386 | @end example | |
3387 | ||
3388 | If the grammar file does not use the @samp{@@} constructs to refer to | |
3389 | textual positions, then the type @code{YYLTYPE} will not be defined. In | |
3390 | this case, omit the second argument; @code{yylex} will be called with | |
3391 | only one argument. | |
3392 | ||
c656404a | 3393 | @vindex YYPARSE_PARAM |
931c7513 RS |
3394 | If you use a reentrant parser, you can optionally pass additional |
3395 | parameter information to it in a reentrant way. To do so, define the | |
3396 | macro @code{YYPARSE_PARAM} as a variable name. This modifies the | |
3397 | @code{yyparse} function to accept one argument, of type @code{void *}, | |
3398 | with that name. | |
e425e872 RS |
3399 | |
3400 | When you call @code{yyparse}, pass the address of an object, casting the | |
3401 | address to @code{void *}. The grammar actions can refer to the contents | |
3402 | of the object by casting the pointer value back to its proper type and | |
3403 | then dereferencing it. Here's an example. Write this in the parser: | |
3404 | ||
3405 | @example | |
3406 | %@{ | |
3407 | struct parser_control | |
3408 | @{ | |
3409 | int nastiness; | |
3410 | int randomness; | |
3411 | @}; | |
3412 | ||
3413 | #define YYPARSE_PARAM parm | |
3414 | %@} | |
3415 | @end example | |
3416 | ||
3417 | @noindent | |
3418 | Then call the parser like this: | |
3419 | ||
3420 | @example | |
3421 | struct parser_control | |
3422 | @{ | |
3423 | int nastiness; | |
3424 | int randomness; | |
3425 | @}; | |
3426 | ||
3427 | @dots{} | |
3428 | ||
3429 | @{ | |
3430 | struct parser_control foo; | |
3431 | @dots{} /* @r{Store proper data in @code{foo}.} */ | |
3432 | value = yyparse ((void *) &foo); | |
3433 | @dots{} | |
3434 | @} | |
3435 | @end example | |
3436 | ||
3437 | @noindent | |
3438 | In the grammar actions, use expressions like this to refer to the data: | |
3439 | ||
3440 | @example | |
3441 | ((struct parser_control *) parm)->randomness | |
3442 | @end example | |
3443 | ||
c656404a RS |
3444 | @vindex YYLEX_PARAM |
3445 | If you wish to pass the additional parameter data to @code{yylex}, | |
3446 | define the macro @code{YYLEX_PARAM} just like @code{YYPARSE_PARAM}, as | |
3447 | shown here: | |
3448 | ||
3449 | @example | |
3450 | %@{ | |
3451 | struct parser_control | |
3452 | @{ | |
3453 | int nastiness; | |
3454 | int randomness; | |
3455 | @}; | |
3456 | ||
3457 | #define YYPARSE_PARAM parm | |
3458 | #define YYLEX_PARAM parm | |
3459 | %@} | |
3460 | @end example | |
3461 | ||
3462 | You should then define @code{yylex} to accept one additional | |
3463 | argument---the value of @code{parm}. (This makes either two or three | |
3464 | arguments in total, depending on whether an argument of type | |
3465 | @code{YYLTYPE} is passed.) You can declare the argument as a pointer to | |
3466 | the proper object type, or you can declare it as @code{void *} and | |
3467 | access the contents as shown above. | |
3468 | ||
931c7513 RS |
3469 | You can use @samp{%pure_parser} to request a reentrant parser without |
3470 | also using @code{YYPARSE_PARAM}. Then you should call @code{yyparse} | |
3471 | with no arguments, as usual. | |
3472 | ||
bfa74976 RS |
3473 | @node Error Reporting, Action Features, Lexical, Interface |
3474 | @section The Error Reporting Function @code{yyerror} | |
3475 | @cindex error reporting function | |
3476 | @findex yyerror | |
3477 | @cindex parse error | |
3478 | @cindex syntax error | |
3479 | ||
3480 | The Bison parser detects a @dfn{parse error} or @dfn{syntax error} | |
9ecbd125 | 3481 | whenever it reads a token which cannot satisfy any syntax rule. An |
bfa74976 | 3482 | action in the grammar can also explicitly proclaim an error, using the |
ceed8467 AD |
3483 | macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use |
3484 | in Actions}). | |
bfa74976 RS |
3485 | |
3486 | The Bison parser expects to report the error by calling an error | |
3487 | reporting function named @code{yyerror}, which you must supply. It is | |
3488 | called by @code{yyparse} whenever a syntax error is found, and it | |
3489 | receives one argument. For a parse error, the string is normally | |
3490 | @w{@code{"parse error"}}. | |
3491 | ||
3492 | @findex YYERROR_VERBOSE | |
3493 | If you define the macro @code{YYERROR_VERBOSE} in the Bison declarations | |
ceed8467 AD |
3494 | section (@pxref{Bison Declarations, ,The Bison Declarations Section}), |
3495 | then Bison provides a more verbose and specific error message string | |
3496 | instead of just plain @w{@code{"parse error"}}. It doesn't matter what | |
3497 | definition you use for @code{YYERROR_VERBOSE}, just whether you define | |
3498 | it. | |
bfa74976 RS |
3499 | |
3500 | The parser can detect one other kind of error: stack overflow. This | |
3501 | happens when the input contains constructions that are very deeply | |
3502 | nested. It isn't likely you will encounter this, since the Bison | |
3503 | parser extends its stack automatically up to a very large limit. But | |
3504 | if overflow happens, @code{yyparse} calls @code{yyerror} in the usual | |
3505 | fashion, except that the argument string is @w{@code{"parser stack | |
3506 | overflow"}}. | |
3507 | ||
3508 | The following definition suffices in simple programs: | |
3509 | ||
3510 | @example | |
3511 | @group | |
13863333 AD |
3512 | void |
3513 | yyerror (char *s) | |
bfa74976 RS |
3514 | @{ |
3515 | @end group | |
3516 | @group | |
3517 | fprintf (stderr, "%s\n", s); | |
3518 | @} | |
3519 | @end group | |
3520 | @end example | |
3521 | ||
3522 | After @code{yyerror} returns to @code{yyparse}, the latter will attempt | |
3523 | error recovery if you have written suitable error recovery grammar rules | |
3524 | (@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will | |
3525 | immediately return 1. | |
3526 | ||
3527 | @vindex yynerrs | |
3528 | The variable @code{yynerrs} contains the number of syntax errors | |
3529 | encountered so far. Normally this variable is global; but if you | |
3530 | request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}) then it is a local variable | |
3531 | which only the actions can access. | |
3532 | ||
3533 | @node Action Features, , Error Reporting, Interface | |
3534 | @section Special Features for Use in Actions | |
3535 | @cindex summary, action features | |
3536 | @cindex action features summary | |
3537 | ||
3538 | Here is a table of Bison constructs, variables and macros that | |
3539 | are useful in actions. | |
3540 | ||
3541 | @table @samp | |
3542 | @item $$ | |
3543 | Acts like a variable that contains the semantic value for the | |
3544 | grouping made by the current rule. @xref{Actions}. | |
3545 | ||
3546 | @item $@var{n} | |
3547 | Acts like a variable that contains the semantic value for the | |
3548 | @var{n}th component of the current rule. @xref{Actions}. | |
3549 | ||
3550 | @item $<@var{typealt}>$ | |
3551 | Like @code{$$} but specifies alternative @var{typealt} in the union | |
3552 | specified by the @code{%union} declaration. @xref{Action Types, ,Data Types of Values in Actions}. | |
3553 | ||
3554 | @item $<@var{typealt}>@var{n} | |
3555 | Like @code{$@var{n}} but specifies alternative @var{typealt} in the | |
13863333 | 3556 | union specified by the @code{%union} declaration. |
bfa74976 RS |
3557 | @xref{Action Types, ,Data Types of Values in Actions}.@refill |
3558 | ||
3559 | @item YYABORT; | |
3560 | Return immediately from @code{yyparse}, indicating failure. | |
3561 | @xref{Parser Function, ,The Parser Function @code{yyparse}}. | |
3562 | ||
3563 | @item YYACCEPT; | |
3564 | Return immediately from @code{yyparse}, indicating success. | |
3565 | @xref{Parser Function, ,The Parser Function @code{yyparse}}. | |
3566 | ||
3567 | @item YYBACKUP (@var{token}, @var{value}); | |
3568 | @findex YYBACKUP | |
3569 | Unshift a token. This macro is allowed only for rules that reduce | |
3570 | a single value, and only when there is no look-ahead token. | |
3571 | It installs a look-ahead token with token type @var{token} and | |
3572 | semantic value @var{value}; then it discards the value that was | |
3573 | going to be reduced by this rule. | |
3574 | ||
3575 | If the macro is used when it is not valid, such as when there is | |
3576 | a look-ahead token already, then it reports a syntax error with | |
3577 | a message @samp{cannot back up} and performs ordinary error | |
3578 | recovery. | |
3579 | ||
3580 | In either case, the rest of the action is not executed. | |
3581 | ||
3582 | @item YYEMPTY | |
3583 | @vindex YYEMPTY | |
3584 | Value stored in @code{yychar} when there is no look-ahead token. | |
3585 | ||
3586 | @item YYERROR; | |
3587 | @findex YYERROR | |
3588 | Cause an immediate syntax error. This statement initiates error | |
3589 | recovery just as if the parser itself had detected an error; however, it | |
3590 | does not call @code{yyerror}, and does not print any message. If you | |
3591 | want to print an error message, call @code{yyerror} explicitly before | |
3592 | the @samp{YYERROR;} statement. @xref{Error Recovery}. | |
3593 | ||
3594 | @item YYRECOVERING | |
3595 | This macro stands for an expression that has the value 1 when the parser | |
3596 | is recovering from a syntax error, and 0 the rest of the time. | |
3597 | @xref{Error Recovery}. | |
3598 | ||
3599 | @item yychar | |
3600 | Variable containing the current look-ahead token. (In a pure parser, | |
3601 | this is actually a local variable within @code{yyparse}.) When there is | |
3602 | no look-ahead token, the value @code{YYEMPTY} is stored in the variable. | |
3603 | @xref{Look-Ahead, ,Look-Ahead Tokens}. | |
3604 | ||
3605 | @item yyclearin; | |
3606 | Discard the current look-ahead token. This is useful primarily in | |
3607 | error rules. @xref{Error Recovery}. | |
3608 | ||
3609 | @item yyerrok; | |
3610 | Resume generating error messages immediately for subsequent syntax | |
13863333 | 3611 | errors. This is useful primarily in error rules. |
bfa74976 RS |
3612 | @xref{Error Recovery}. |
3613 | ||
847bf1f5 AD |
3614 | @item @@$ |
3615 | @findex @@$ | |
3616 | Acts like a structure variable containing information on the textual position | |
3617 | of the grouping made by the current rule. @xref{Locations, , | |
3618 | Tracking Locations}. | |
bfa74976 | 3619 | |
847bf1f5 AD |
3620 | @c Check if those paragraphs are still useful or not. |
3621 | ||
3622 | @c @example | |
3623 | @c struct @{ | |
3624 | @c int first_line, last_line; | |
3625 | @c int first_column, last_column; | |
3626 | @c @}; | |
3627 | @c @end example | |
3628 | ||
3629 | @c Thus, to get the starting line number of the third component, you would | |
3630 | @c use @samp{@@3.first_line}. | |
bfa74976 | 3631 | |
847bf1f5 AD |
3632 | @c In order for the members of this structure to contain valid information, |
3633 | @c you must make @code{yylex} supply this information about each token. | |
3634 | @c If you need only certain members, then @code{yylex} need only fill in | |
3635 | @c those members. | |
bfa74976 | 3636 | |
847bf1f5 AD |
3637 | @c The use of this feature makes the parser noticeably slower. |
3638 | ||
3639 | @item @@@var{n} | |
3640 | @findex @@@var{n} | |
3641 | Acts like a structure variable containing information on the textual position | |
3642 | of the @var{n}th component of the current rule. @xref{Locations, , | |
3643 | Tracking Locations}. | |
bfa74976 | 3644 | |
bfa74976 RS |
3645 | @end table |
3646 | ||
3647 | @node Algorithm, Error Recovery, Interface, Top | |
13863333 AD |
3648 | @chapter The Bison Parser Algorithm |
3649 | @cindex Bison parser algorithm | |
bfa74976 RS |
3650 | @cindex algorithm of parser |
3651 | @cindex shifting | |
3652 | @cindex reduction | |
3653 | @cindex parser stack | |
3654 | @cindex stack, parser | |
3655 | ||
3656 | As Bison reads tokens, it pushes them onto a stack along with their | |
3657 | semantic values. The stack is called the @dfn{parser stack}. Pushing a | |
3658 | token is traditionally called @dfn{shifting}. | |
3659 | ||
3660 | For example, suppose the infix calculator has read @samp{1 + 5 *}, with a | |
3661 | @samp{3} to come. The stack will have four elements, one for each token | |
3662 | that was shifted. | |
3663 | ||
3664 | But the stack does not always have an element for each token read. When | |
3665 | the last @var{n} tokens and groupings shifted match the components of a | |
3666 | grammar rule, they can be combined according to that rule. This is called | |
3667 | @dfn{reduction}. Those tokens and groupings are replaced on the stack by a | |
3668 | single grouping whose symbol is the result (left hand side) of that rule. | |
3669 | Running the rule's action is part of the process of reduction, because this | |
3670 | is what computes the semantic value of the resulting grouping. | |
3671 | ||
3672 | For example, if the infix calculator's parser stack contains this: | |
3673 | ||
3674 | @example | |
3675 | 1 + 5 * 3 | |
3676 | @end example | |
3677 | ||
3678 | @noindent | |
3679 | and the next input token is a newline character, then the last three | |
3680 | elements can be reduced to 15 via the rule: | |
3681 | ||
3682 | @example | |
3683 | expr: expr '*' expr; | |
3684 | @end example | |
3685 | ||
3686 | @noindent | |
3687 | Then the stack contains just these three elements: | |
3688 | ||
3689 | @example | |
3690 | 1 + 15 | |
3691 | @end example | |
3692 | ||
3693 | @noindent | |
3694 | At this point, another reduction can be made, resulting in the single value | |
3695 | 16. Then the newline token can be shifted. | |
3696 | ||
3697 | The parser tries, by shifts and reductions, to reduce the entire input down | |
3698 | to a single grouping whose symbol is the grammar's start-symbol | |
3699 | (@pxref{Language and Grammar, ,Languages and Context-Free Grammars}). | |
3700 | ||
3701 | This kind of parser is known in the literature as a bottom-up parser. | |
3702 | ||
3703 | @menu | |
3704 | * Look-Ahead:: Parser looks one token ahead when deciding what to do. | |
3705 | * Shift/Reduce:: Conflicts: when either shifting or reduction is valid. | |
3706 | * Precedence:: Operator precedence works by resolving conflicts. | |
3707 | * Contextual Precedence:: When an operator's precedence depends on context. | |
3708 | * Parser States:: The parser is a finite-state-machine with stack. | |
3709 | * Reduce/Reduce:: When two rules are applicable in the same situation. | |
3710 | * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified. | |
3711 | * Stack Overflow:: What happens when stack gets full. How to avoid it. | |
3712 | @end menu | |
3713 | ||
3714 | @node Look-Ahead, Shift/Reduce, , Algorithm | |
3715 | @section Look-Ahead Tokens | |
3716 | @cindex look-ahead token | |
3717 | ||
3718 | The Bison parser does @emph{not} always reduce immediately as soon as the | |
3719 | last @var{n} tokens and groupings match a rule. This is because such a | |
3720 | simple strategy is inadequate to handle most languages. Instead, when a | |
3721 | reduction is possible, the parser sometimes ``looks ahead'' at the next | |
3722 | token in order to decide what to do. | |
3723 | ||
3724 | When a token is read, it is not immediately shifted; first it becomes the | |
3725 | @dfn{look-ahead token}, which is not on the stack. Now the parser can | |
3726 | perform one or more reductions of tokens and groupings on the stack, while | |
3727 | the look-ahead token remains off to the side. When no more reductions | |
3728 | should take place, the look-ahead token is shifted onto the stack. This | |
3729 | does not mean that all possible reductions have been done; depending on the | |
3730 | token type of the look-ahead token, some rules may choose to delay their | |
3731 | application. | |
3732 | ||
3733 | Here is a simple case where look-ahead is needed. These three rules define | |
3734 | expressions which contain binary addition operators and postfix unary | |
3735 | factorial operators (@samp{!}), and allow parentheses for grouping. | |
3736 | ||
3737 | @example | |
3738 | @group | |
3739 | expr: term '+' expr | |
3740 | | term | |
3741 | ; | |
3742 | @end group | |
3743 | ||
3744 | @group | |
3745 | term: '(' expr ')' | |
3746 | | term '!' | |
3747 | | NUMBER | |
3748 | ; | |
3749 | @end group | |
3750 | @end example | |
3751 | ||
3752 | Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what | |
3753 | should be done? If the following token is @samp{)}, then the first three | |
3754 | tokens must be reduced to form an @code{expr}. This is the only valid | |
3755 | course, because shifting the @samp{)} would produce a sequence of symbols | |
3756 | @w{@code{term ')'}}, and no rule allows this. | |
3757 | ||
3758 | If the following token is @samp{!}, then it must be shifted immediately so | |
3759 | that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the | |
3760 | parser were to reduce before shifting, @w{@samp{1 + 2}} would become an | |
3761 | @code{expr}. It would then be impossible to shift the @samp{!} because | |
3762 | doing so would produce on the stack the sequence of symbols @code{expr | |
3763 | '!'}. No rule allows that sequence. | |
3764 | ||
3765 | @vindex yychar | |
3766 | The current look-ahead token is stored in the variable @code{yychar}. | |
3767 | @xref{Action Features, ,Special Features for Use in Actions}. | |
3768 | ||
3769 | @node Shift/Reduce, Precedence, Look-Ahead, Algorithm | |
3770 | @section Shift/Reduce Conflicts | |
3771 | @cindex conflicts | |
3772 | @cindex shift/reduce conflicts | |
3773 | @cindex dangling @code{else} | |
3774 | @cindex @code{else}, dangling | |
3775 | ||
3776 | Suppose we are parsing a language which has if-then and if-then-else | |
3777 | statements, with a pair of rules like this: | |
3778 | ||
3779 | @example | |
3780 | @group | |
3781 | if_stmt: | |
3782 | IF expr THEN stmt | |
3783 | | IF expr THEN stmt ELSE stmt | |
3784 | ; | |
3785 | @end group | |
3786 | @end example | |
3787 | ||
3788 | @noindent | |
3789 | Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are | |
3790 | terminal symbols for specific keyword tokens. | |
3791 | ||
3792 | When the @code{ELSE} token is read and becomes the look-ahead token, the | |
3793 | contents of the stack (assuming the input is valid) are just right for | |
3794 | reduction by the first rule. But it is also legitimate to shift the | |
3795 | @code{ELSE}, because that would lead to eventual reduction by the second | |
3796 | rule. | |
3797 | ||
3798 | This situation, where either a shift or a reduction would be valid, is | |
3799 | called a @dfn{shift/reduce conflict}. Bison is designed to resolve | |
3800 | these conflicts by choosing to shift, unless otherwise directed by | |
3801 | operator precedence declarations. To see the reason for this, let's | |
3802 | contrast it with the other alternative. | |
3803 | ||
3804 | Since the parser prefers to shift the @code{ELSE}, the result is to attach | |
3805 | the else-clause to the innermost if-statement, making these two inputs | |
3806 | equivalent: | |
3807 | ||
3808 | @example | |
3809 | if x then if y then win (); else lose; | |
3810 | ||
3811 | if x then do; if y then win (); else lose; end; | |
3812 | @end example | |
3813 | ||
3814 | But if the parser chose to reduce when possible rather than shift, the | |
3815 | result would be to attach the else-clause to the outermost if-statement, | |
3816 | making these two inputs equivalent: | |
3817 | ||
3818 | @example | |
3819 | if x then if y then win (); else lose; | |
3820 | ||
3821 | if x then do; if y then win (); end; else lose; | |
3822 | @end example | |
3823 | ||
3824 | The conflict exists because the grammar as written is ambiguous: either | |
3825 | parsing of the simple nested if-statement is legitimate. The established | |
3826 | convention is that these ambiguities are resolved by attaching the | |
3827 | else-clause to the innermost if-statement; this is what Bison accomplishes | |
3828 | by choosing to shift rather than reduce. (It would ideally be cleaner to | |
3829 | write an unambiguous grammar, but that is very hard to do in this case.) | |
3830 | This particular ambiguity was first encountered in the specifications of | |
3831 | Algol 60 and is called the ``dangling @code{else}'' ambiguity. | |
3832 | ||
3833 | To avoid warnings from Bison about predictable, legitimate shift/reduce | |
3834 | conflicts, use the @code{%expect @var{n}} declaration. There will be no | |
3835 | warning as long as the number of shift/reduce conflicts is exactly @var{n}. | |
3836 | @xref{Expect Decl, ,Suppressing Conflict Warnings}. | |
3837 | ||
3838 | The definition of @code{if_stmt} above is solely to blame for the | |
3839 | conflict, but the conflict does not actually appear without additional | |
3840 | rules. Here is a complete Bison input file that actually manifests the | |
3841 | conflict: | |
3842 | ||
3843 | @example | |
3844 | @group | |
3845 | %token IF THEN ELSE variable | |
3846 | %% | |
3847 | @end group | |
3848 | @group | |
3849 | stmt: expr | |
3850 | | if_stmt | |
3851 | ; | |
3852 | @end group | |
3853 | ||
3854 | @group | |
3855 | if_stmt: | |
3856 | IF expr THEN stmt | |
3857 | | IF expr THEN stmt ELSE stmt | |
3858 | ; | |
3859 | @end group | |
3860 | ||
3861 | expr: variable | |
3862 | ; | |
3863 | @end example | |
3864 | ||
3865 | @node Precedence, Contextual Precedence, Shift/Reduce, Algorithm | |
3866 | @section Operator Precedence | |
3867 | @cindex operator precedence | |
3868 | @cindex precedence of operators | |
3869 | ||
3870 | Another situation where shift/reduce conflicts appear is in arithmetic | |
3871 | expressions. Here shifting is not always the preferred resolution; the | |
3872 | Bison declarations for operator precedence allow you to specify when to | |
3873 | shift and when to reduce. | |
3874 | ||
3875 | @menu | |
3876 | * Why Precedence:: An example showing why precedence is needed. | |
3877 | * Using Precedence:: How to specify precedence in Bison grammars. | |
3878 | * Precedence Examples:: How these features are used in the previous example. | |
3879 | * How Precedence:: How they work. | |
3880 | @end menu | |
3881 | ||
3882 | @node Why Precedence, Using Precedence, , Precedence | |
3883 | @subsection When Precedence is Needed | |
3884 | ||
3885 | Consider the following ambiguous grammar fragment (ambiguous because the | |
3886 | input @w{@samp{1 - 2 * 3}} can be parsed in two different ways): | |
3887 | ||
3888 | @example | |
3889 | @group | |
3890 | expr: expr '-' expr | |
3891 | | expr '*' expr | |
3892 | | expr '<' expr | |
3893 | | '(' expr ')' | |
3894 | @dots{} | |
3895 | ; | |
3896 | @end group | |
3897 | @end example | |
3898 | ||
3899 | @noindent | |
3900 | Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2}; | |
14ded682 AD |
3901 | should it reduce them via the rule for the subtraction operator? It |
3902 | depends on the next token. Of course, if the next token is @samp{)}, we | |
3903 | must reduce; shifting is invalid because no single rule can reduce the | |
3904 | token sequence @w{@samp{- 2 )}} or anything starting with that. But if | |
3905 | the next token is @samp{*} or @samp{<}, we have a choice: either | |
3906 | shifting or reduction would allow the parse to complete, but with | |
3907 | different results. | |
3908 | ||
3909 | To decide which one Bison should do, we must consider the results. If | |
3910 | the next operator token @var{op} is shifted, then it must be reduced | |
3911 | first in order to permit another opportunity to reduce the difference. | |
3912 | The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other | |
3913 | hand, if the subtraction is reduced before shifting @var{op}, the result | |
3914 | is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or | |
3915 | reduce should depend on the relative precedence of the operators | |
3916 | @samp{-} and @var{op}: @samp{*} should be shifted first, but not | |
3917 | @samp{<}. | |
bfa74976 RS |
3918 | |
3919 | @cindex associativity | |
3920 | What about input such as @w{@samp{1 - 2 - 5}}; should this be | |
14ded682 AD |
3921 | @w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most |
3922 | operators we prefer the former, which is called @dfn{left association}. | |
3923 | The latter alternative, @dfn{right association}, is desirable for | |
3924 | assignment operators. The choice of left or right association is a | |
3925 | matter of whether the parser chooses to shift or reduce when the stack | |
3926 | contains @w{@samp{1 - 2}} and the look-ahead token is @samp{-}: shifting | |
3927 | makes right-associativity. | |
bfa74976 RS |
3928 | |
3929 | @node Using Precedence, Precedence Examples, Why Precedence, Precedence | |
3930 | @subsection Specifying Operator Precedence | |
3931 | @findex %left | |
3932 | @findex %right | |
3933 | @findex %nonassoc | |
3934 | ||
3935 | Bison allows you to specify these choices with the operator precedence | |
3936 | declarations @code{%left} and @code{%right}. Each such declaration | |
3937 | contains a list of tokens, which are operators whose precedence and | |
3938 | associativity is being declared. The @code{%left} declaration makes all | |
3939 | those operators left-associative and the @code{%right} declaration makes | |
3940 | them right-associative. A third alternative is @code{%nonassoc}, which | |
3941 | declares that it is a syntax error to find the same operator twice ``in a | |
3942 | row''. | |
3943 | ||
3944 | The relative precedence of different operators is controlled by the | |
3945 | order in which they are declared. The first @code{%left} or | |
3946 | @code{%right} declaration in the file declares the operators whose | |
3947 | precedence is lowest, the next such declaration declares the operators | |
3948 | whose precedence is a little higher, and so on. | |
3949 | ||
3950 | @node Precedence Examples, How Precedence, Using Precedence, Precedence | |
3951 | @subsection Precedence Examples | |
3952 | ||
3953 | In our example, we would want the following declarations: | |
3954 | ||
3955 | @example | |
3956 | %left '<' | |
3957 | %left '-' | |
3958 | %left '*' | |
3959 | @end example | |
3960 | ||
3961 | In a more complete example, which supports other operators as well, we | |
3962 | would declare them in groups of equal precedence. For example, @code{'+'} is | |
3963 | declared with @code{'-'}: | |
3964 | ||
3965 | @example | |
3966 | %left '<' '>' '=' NE LE GE | |
3967 | %left '+' '-' | |
3968 | %left '*' '/' | |
3969 | @end example | |
3970 | ||
3971 | @noindent | |
3972 | (Here @code{NE} and so on stand for the operators for ``not equal'' | |
3973 | and so on. We assume that these tokens are more than one character long | |
3974 | and therefore are represented by names, not character literals.) | |
3975 | ||
3976 | @node How Precedence, , Precedence Examples, Precedence | |
3977 | @subsection How Precedence Works | |
3978 | ||
3979 | The first effect of the precedence declarations is to assign precedence | |
3980 | levels to the terminal symbols declared. The second effect is to assign | |
3981 | precedence levels to certain rules: each rule gets its precedence from the | |
3982 | last terminal symbol mentioned in the components. (You can also specify | |
3983 | explicitly the precedence of a rule. @xref{Contextual Precedence, ,Context-Dependent Precedence}.) | |
3984 | ||
3985 | Finally, the resolution of conflicts works by comparing the | |
3986 | precedence of the rule being considered with that of the | |
3987 | look-ahead token. If the token's precedence is higher, the | |
3988 | choice is to shift. If the rule's precedence is higher, the | |
3989 | choice is to reduce. If they have equal precedence, the choice | |
3990 | is made based on the associativity of that precedence level. The | |
3991 | verbose output file made by @samp{-v} (@pxref{Invocation, ,Invoking Bison}) says | |
3992 | how each conflict was resolved. | |
3993 | ||
3994 | Not all rules and not all tokens have precedence. If either the rule or | |
3995 | the look-ahead token has no precedence, then the default is to shift. | |
3996 | ||
3997 | @node Contextual Precedence, Parser States, Precedence, Algorithm | |
3998 | @section Context-Dependent Precedence | |
3999 | @cindex context-dependent precedence | |
4000 | @cindex unary operator precedence | |
4001 | @cindex precedence, context-dependent | |
4002 | @cindex precedence, unary operator | |
4003 | @findex %prec | |
4004 | ||
4005 | Often the precedence of an operator depends on the context. This sounds | |
4006 | outlandish at first, but it is really very common. For example, a minus | |
4007 | sign typically has a very high precedence as a unary operator, and a | |
4008 | somewhat lower precedence (lower than multiplication) as a binary operator. | |
4009 | ||
4010 | The Bison precedence declarations, @code{%left}, @code{%right} and | |
4011 | @code{%nonassoc}, can only be used once for a given token; so a token has | |
4012 | only one precedence declared in this way. For context-dependent | |
4013 | precedence, you need to use an additional mechanism: the @code{%prec} | |
4014 | modifier for rules.@refill | |
4015 | ||
4016 | The @code{%prec} modifier declares the precedence of a particular rule by | |
4017 | specifying a terminal symbol whose precedence should be used for that rule. | |
4018 | It's not necessary for that symbol to appear otherwise in the rule. The | |
4019 | modifier's syntax is: | |
4020 | ||
4021 | @example | |
4022 | %prec @var{terminal-symbol} | |
4023 | @end example | |
4024 | ||
4025 | @noindent | |
4026 | and it is written after the components of the rule. Its effect is to | |
4027 | assign the rule the precedence of @var{terminal-symbol}, overriding | |
4028 | the precedence that would be deduced for it in the ordinary way. The | |
4029 | altered rule precedence then affects how conflicts involving that rule | |
4030 | are resolved (@pxref{Precedence, ,Operator Precedence}). | |
4031 | ||
4032 | Here is how @code{%prec} solves the problem of unary minus. First, declare | |
4033 | a precedence for a fictitious terminal symbol named @code{UMINUS}. There | |
4034 | are no tokens of this type, but the symbol serves to stand for its | |
4035 | precedence: | |
4036 | ||
4037 | @example | |
4038 | @dots{} | |
4039 | %left '+' '-' | |
4040 | %left '*' | |
4041 | %left UMINUS | |
4042 | @end example | |
4043 | ||
4044 | Now the precedence of @code{UMINUS} can be used in specific rules: | |
4045 | ||
4046 | @example | |
4047 | @group | |
4048 | exp: @dots{} | |
4049 | | exp '-' exp | |
4050 | @dots{} | |
4051 | | '-' exp %prec UMINUS | |
4052 | @end group | |
4053 | @end example | |
4054 | ||
4055 | @node Parser States, Reduce/Reduce, Contextual Precedence, Algorithm | |
4056 | @section Parser States | |
4057 | @cindex finite-state machine | |
4058 | @cindex parser state | |
4059 | @cindex state (of parser) | |
4060 | ||
4061 | The function @code{yyparse} is implemented using a finite-state machine. | |
4062 | The values pushed on the parser stack are not simply token type codes; they | |
4063 | represent the entire sequence of terminal and nonterminal symbols at or | |
4064 | near the top of the stack. The current state collects all the information | |
4065 | about previous input which is relevant to deciding what to do next. | |
4066 | ||
4067 | Each time a look-ahead token is read, the current parser state together | |
4068 | with the type of look-ahead token are looked up in a table. This table | |
4069 | entry can say, ``Shift the look-ahead token.'' In this case, it also | |
4070 | specifies the new parser state, which is pushed onto the top of the | |
4071 | parser stack. Or it can say, ``Reduce using rule number @var{n}.'' | |
4072 | This means that a certain number of tokens or groupings are taken off | |
4073 | the top of the stack, and replaced by one grouping. In other words, | |
4074 | that number of states are popped from the stack, and one new state is | |
4075 | pushed. | |
4076 | ||
4077 | There is one other alternative: the table can say that the look-ahead token | |
4078 | is erroneous in the current state. This causes error processing to begin | |
4079 | (@pxref{Error Recovery}). | |
4080 | ||
4081 | @node Reduce/Reduce, Mystery Conflicts, Parser States, Algorithm | |
4082 | @section Reduce/Reduce Conflicts | |
4083 | @cindex reduce/reduce conflict | |
4084 | @cindex conflicts, reduce/reduce | |
4085 | ||
4086 | A reduce/reduce conflict occurs if there are two or more rules that apply | |
4087 | to the same sequence of input. This usually indicates a serious error | |
4088 | in the grammar. | |
4089 | ||
4090 | For example, here is an erroneous attempt to define a sequence | |
4091 | of zero or more @code{word} groupings. | |
4092 | ||
4093 | @example | |
4094 | sequence: /* empty */ | |
4095 | @{ printf ("empty sequence\n"); @} | |
4096 | | maybeword | |
4097 | | sequence word | |
4098 | @{ printf ("added word %s\n", $2); @} | |
4099 | ; | |
4100 | ||
4101 | maybeword: /* empty */ | |
4102 | @{ printf ("empty maybeword\n"); @} | |
4103 | | word | |
4104 | @{ printf ("single word %s\n", $1); @} | |
4105 | ; | |
4106 | @end example | |
4107 | ||
4108 | @noindent | |
4109 | The error is an ambiguity: there is more than one way to parse a single | |
4110 | @code{word} into a @code{sequence}. It could be reduced to a | |
4111 | @code{maybeword} and then into a @code{sequence} via the second rule. | |
4112 | Alternatively, nothing-at-all could be reduced into a @code{sequence} | |
4113 | via the first rule, and this could be combined with the @code{word} | |
4114 | using the third rule for @code{sequence}. | |
4115 | ||
4116 | There is also more than one way to reduce nothing-at-all into a | |
4117 | @code{sequence}. This can be done directly via the first rule, | |
4118 | or indirectly via @code{maybeword} and then the second rule. | |
4119 | ||
4120 | You might think that this is a distinction without a difference, because it | |
4121 | does not change whether any particular input is valid or not. But it does | |
4122 | affect which actions are run. One parsing order runs the second rule's | |
4123 | action; the other runs the first rule's action and the third rule's action. | |
4124 | In this example, the output of the program changes. | |
4125 | ||
4126 | Bison resolves a reduce/reduce conflict by choosing to use the rule that | |
4127 | appears first in the grammar, but it is very risky to rely on this. Every | |
4128 | reduce/reduce conflict must be studied and usually eliminated. Here is the | |
4129 | proper way to define @code{sequence}: | |
4130 | ||
4131 | @example | |
4132 | sequence: /* empty */ | |
4133 | @{ printf ("empty sequence\n"); @} | |
4134 | | sequence word | |
4135 | @{ printf ("added word %s\n", $2); @} | |
4136 | ; | |
4137 | @end example | |
4138 | ||
4139 | Here is another common error that yields a reduce/reduce conflict: | |
4140 | ||
4141 | @example | |
4142 | sequence: /* empty */ | |
4143 | | sequence words | |
4144 | | sequence redirects | |
4145 | ; | |
4146 | ||
4147 | words: /* empty */ | |
4148 | | words word | |
4149 | ; | |
4150 | ||
4151 | redirects:/* empty */ | |
4152 | | redirects redirect | |
4153 | ; | |
4154 | @end example | |
4155 | ||
4156 | @noindent | |
4157 | The intention here is to define a sequence which can contain either | |
4158 | @code{word} or @code{redirect} groupings. The individual definitions of | |
4159 | @code{sequence}, @code{words} and @code{redirects} are error-free, but the | |
4160 | three together make a subtle ambiguity: even an empty input can be parsed | |
4161 | in infinitely many ways! | |
4162 | ||
4163 | Consider: nothing-at-all could be a @code{words}. Or it could be two | |
4164 | @code{words} in a row, or three, or any number. It could equally well be a | |
4165 | @code{redirects}, or two, or any number. Or it could be a @code{words} | |
4166 | followed by three @code{redirects} and another @code{words}. And so on. | |
4167 | ||
4168 | Here are two ways to correct these rules. First, to make it a single level | |
4169 | of sequence: | |
4170 | ||
4171 | @example | |
4172 | sequence: /* empty */ | |
4173 | | sequence word | |
4174 | | sequence redirect | |
4175 | ; | |
4176 | @end example | |
4177 | ||
4178 | Second, to prevent either a @code{words} or a @code{redirects} | |
4179 | from being empty: | |
4180 | ||
4181 | @example | |
4182 | sequence: /* empty */ | |
4183 | | sequence words | |
4184 | | sequence redirects | |
4185 | ; | |
4186 | ||
4187 | words: word | |
4188 | | words word | |
4189 | ; | |
4190 | ||
4191 | redirects:redirect | |
4192 | | redirects redirect | |
4193 | ; | |
4194 | @end example | |
4195 | ||
4196 | @node Mystery Conflicts, Stack Overflow, Reduce/Reduce, Algorithm | |
4197 | @section Mysterious Reduce/Reduce Conflicts | |
4198 | ||
4199 | Sometimes reduce/reduce conflicts can occur that don't look warranted. | |
4200 | Here is an example: | |
4201 | ||
4202 | @example | |
4203 | @group | |
4204 | %token ID | |
4205 | ||
4206 | %% | |
4207 | def: param_spec return_spec ',' | |
4208 | ; | |
4209 | param_spec: | |
4210 | type | |
4211 | | name_list ':' type | |
4212 | ; | |
4213 | @end group | |
4214 | @group | |
4215 | return_spec: | |
4216 | type | |
4217 | | name ':' type | |
4218 | ; | |
4219 | @end group | |
4220 | @group | |
4221 | type: ID | |
4222 | ; | |
4223 | @end group | |
4224 | @group | |
4225 | name: ID | |
4226 | ; | |
4227 | name_list: | |
4228 | name | |
4229 | | name ',' name_list | |
4230 | ; | |
4231 | @end group | |
4232 | @end example | |
4233 | ||
4234 | It would seem that this grammar can be parsed with only a single token | |
13863333 | 4235 | of look-ahead: when a @code{param_spec} is being read, an @code{ID} is |
bfa74976 RS |
4236 | a @code{name} if a comma or colon follows, or a @code{type} if another |
4237 | @code{ID} follows. In other words, this grammar is LR(1). | |
4238 | ||
4239 | @cindex LR(1) | |
4240 | @cindex LALR(1) | |
4241 | However, Bison, like most parser generators, cannot actually handle all | |
4242 | LR(1) grammars. In this grammar, two contexts, that after an @code{ID} | |
4243 | at the beginning of a @code{param_spec} and likewise at the beginning of | |
4244 | a @code{return_spec}, are similar enough that Bison assumes they are the | |
4245 | same. They appear similar because the same set of rules would be | |
4246 | active---the rule for reducing to a @code{name} and that for reducing to | |
4247 | a @code{type}. Bison is unable to determine at that stage of processing | |
4248 | that the rules would require different look-ahead tokens in the two | |
4249 | contexts, so it makes a single parser state for them both. Combining | |
4250 | the two contexts causes a conflict later. In parser terminology, this | |
4251 | occurrence means that the grammar is not LALR(1). | |
4252 | ||
4253 | In general, it is better to fix deficiencies than to document them. But | |
4254 | this particular deficiency is intrinsically hard to fix; parser | |
4255 | generators that can handle LR(1) grammars are hard to write and tend to | |
4256 | produce parsers that are very large. In practice, Bison is more useful | |
4257 | as it is now. | |
4258 | ||
4259 | When the problem arises, you can often fix it by identifying the two | |
4260 | parser states that are being confused, and adding something to make them | |
4261 | look distinct. In the above example, adding one rule to | |
4262 | @code{return_spec} as follows makes the problem go away: | |
4263 | ||
4264 | @example | |
4265 | @group | |
4266 | %token BOGUS | |
4267 | @dots{} | |
4268 | %% | |
4269 | @dots{} | |
4270 | return_spec: | |
4271 | type | |
4272 | | name ':' type | |
4273 | /* This rule is never used. */ | |
4274 | | ID BOGUS | |
4275 | ; | |
4276 | @end group | |
4277 | @end example | |
4278 | ||
4279 | This corrects the problem because it introduces the possibility of an | |
4280 | additional active rule in the context after the @code{ID} at the beginning of | |
4281 | @code{return_spec}. This rule is not active in the corresponding context | |
4282 | in a @code{param_spec}, so the two contexts receive distinct parser states. | |
4283 | As long as the token @code{BOGUS} is never generated by @code{yylex}, | |
4284 | the added rule cannot alter the way actual input is parsed. | |
4285 | ||
4286 | In this particular example, there is another way to solve the problem: | |
4287 | rewrite the rule for @code{return_spec} to use @code{ID} directly | |
4288 | instead of via @code{name}. This also causes the two confusing | |
4289 | contexts to have different sets of active rules, because the one for | |
4290 | @code{return_spec} activates the altered rule for @code{return_spec} | |
4291 | rather than the one for @code{name}. | |
4292 | ||
4293 | @example | |
4294 | param_spec: | |
4295 | type | |
4296 | | name_list ':' type | |
4297 | ; | |
4298 | return_spec: | |
4299 | type | |
4300 | | ID ':' type | |
4301 | ; | |
4302 | @end example | |
4303 | ||
4304 | @node Stack Overflow, , Mystery Conflicts, Algorithm | |
4305 | @section Stack Overflow, and How to Avoid It | |
4306 | @cindex stack overflow | |
4307 | @cindex parser stack overflow | |
4308 | @cindex overflow of parser stack | |
4309 | ||
4310 | The Bison parser stack can overflow if too many tokens are shifted and | |
4311 | not reduced. When this happens, the parser function @code{yyparse} | |
4312 | returns a nonzero value, pausing only to call @code{yyerror} to report | |
4313 | the overflow. | |
4314 | ||
4315 | @vindex YYMAXDEPTH | |
4316 | By defining the macro @code{YYMAXDEPTH}, you can control how deep the | |
4317 | parser stack can become before a stack overflow occurs. Define the | |
4318 | macro with a value that is an integer. This value is the maximum number | |
4319 | of tokens that can be shifted (and not reduced) before overflow. | |
4320 | It must be a constant expression whose value is known at compile time. | |
4321 | ||
4322 | The stack space allowed is not necessarily allocated. If you specify a | |
4323 | large value for @code{YYMAXDEPTH}, the parser actually allocates a small | |
4324 | stack at first, and then makes it bigger by stages as needed. This | |
4325 | increasing allocation happens automatically and silently. Therefore, | |
4326 | you do not need to make @code{YYMAXDEPTH} painfully small merely to save | |
4327 | space for ordinary inputs that do not need much stack. | |
4328 | ||
4329 | @cindex default stack limit | |
4330 | The default value of @code{YYMAXDEPTH}, if you do not define it, is | |
4331 | 10000. | |
4332 | ||
4333 | @vindex YYINITDEPTH | |
4334 | You can control how much stack is allocated initially by defining the | |
4335 | macro @code{YYINITDEPTH}. This value too must be a compile-time | |
4336 | constant integer. The default is 200. | |
4337 | ||
4338 | @node Error Recovery, Context Dependency, Algorithm, Top | |
4339 | @chapter Error Recovery | |
4340 | @cindex error recovery | |
4341 | @cindex recovery from errors | |
4342 | ||
4343 | It is not usually acceptable to have a program terminate on a parse | |
4344 | error. For example, a compiler should recover sufficiently to parse the | |
4345 | rest of the input file and check it for errors; a calculator should accept | |
4346 | another expression. | |
4347 | ||
4348 | In a simple interactive command parser where each input is one line, it may | |
4349 | be sufficient to allow @code{yyparse} to return 1 on error and have the | |
4350 | caller ignore the rest of the input line when that happens (and then call | |
4351 | @code{yyparse} again). But this is inadequate for a compiler, because it | |
4352 | forgets all the syntactic context leading up to the error. A syntax error | |
4353 | deep within a function in the compiler input should not cause the compiler | |
4354 | to treat the following line like the beginning of a source file. | |
4355 | ||
4356 | @findex error | |
4357 | You can define how to recover from a syntax error by writing rules to | |
4358 | recognize the special token @code{error}. This is a terminal symbol that | |
4359 | is always defined (you need not declare it) and reserved for error | |
4360 | handling. The Bison parser generates an @code{error} token whenever a | |
4361 | syntax error happens; if you have provided a rule to recognize this token | |
13863333 | 4362 | in the current context, the parse can continue. |
bfa74976 RS |
4363 | |
4364 | For example: | |
4365 | ||
4366 | @example | |
4367 | stmnts: /* empty string */ | |
4368 | | stmnts '\n' | |
4369 | | stmnts exp '\n' | |
4370 | | stmnts error '\n' | |
4371 | @end example | |
4372 | ||
4373 | The fourth rule in this example says that an error followed by a newline | |
4374 | makes a valid addition to any @code{stmnts}. | |
4375 | ||
4376 | What happens if a syntax error occurs in the middle of an @code{exp}? The | |
4377 | error recovery rule, interpreted strictly, applies to the precise sequence | |
4378 | of a @code{stmnts}, an @code{error} and a newline. If an error occurs in | |
4379 | the middle of an @code{exp}, there will probably be some additional tokens | |
4380 | and subexpressions on the stack after the last @code{stmnts}, and there | |
4381 | will be tokens to read before the next newline. So the rule is not | |
4382 | applicable in the ordinary way. | |
4383 | ||
4384 | But Bison can force the situation to fit the rule, by discarding part of | |
4385 | the semantic context and part of the input. First it discards states and | |
4386 | objects from the stack until it gets back to a state in which the | |
4387 | @code{error} token is acceptable. (This means that the subexpressions | |
4388 | already parsed are discarded, back to the last complete @code{stmnts}.) At | |
4389 | this point the @code{error} token can be shifted. Then, if the old | |
4390 | look-ahead token is not acceptable to be shifted next, the parser reads | |
4391 | tokens and discards them until it finds a token which is acceptable. In | |
4392 | this example, Bison reads and discards input until the next newline | |
4393 | so that the fourth rule can apply. | |
4394 | ||
4395 | The choice of error rules in the grammar is a choice of strategies for | |
4396 | error recovery. A simple and useful strategy is simply to skip the rest of | |
4397 | the current input line or current statement if an error is detected: | |
4398 | ||
4399 | @example | |
4400 | stmnt: error ';' /* on error, skip until ';' is read */ | |
4401 | @end example | |
4402 | ||
4403 | It is also useful to recover to the matching close-delimiter of an | |
4404 | opening-delimiter that has already been parsed. Otherwise the | |
4405 | close-delimiter will probably appear to be unmatched, and generate another, | |
4406 | spurious error message: | |
4407 | ||
4408 | @example | |
4409 | primary: '(' expr ')' | |
4410 | | '(' error ')' | |
4411 | @dots{} | |
4412 | ; | |
4413 | @end example | |
4414 | ||
4415 | Error recovery strategies are necessarily guesses. When they guess wrong, | |
4416 | one syntax error often leads to another. In the above example, the error | |
4417 | recovery rule guesses that an error is due to bad input within one | |
4418 | @code{stmnt}. Suppose that instead a spurious semicolon is inserted in the | |
4419 | middle of a valid @code{stmnt}. After the error recovery rule recovers | |
4420 | from the first error, another syntax error will be found straightaway, | |
4421 | since the text following the spurious semicolon is also an invalid | |
4422 | @code{stmnt}. | |
4423 | ||
4424 | To prevent an outpouring of error messages, the parser will output no error | |
4425 | message for another syntax error that happens shortly after the first; only | |
4426 | after three consecutive input tokens have been successfully shifted will | |
4427 | error messages resume. | |
4428 | ||
4429 | Note that rules which accept the @code{error} token may have actions, just | |
4430 | as any other rules can. | |
4431 | ||
4432 | @findex yyerrok | |
4433 | You can make error messages resume immediately by using the macro | |
4434 | @code{yyerrok} in an action. If you do this in the error rule's action, no | |
4435 | error messages will be suppressed. This macro requires no arguments; | |
4436 | @samp{yyerrok;} is a valid C statement. | |
4437 | ||
4438 | @findex yyclearin | |
4439 | The previous look-ahead token is reanalyzed immediately after an error. If | |
4440 | this is unacceptable, then the macro @code{yyclearin} may be used to clear | |
4441 | this token. Write the statement @samp{yyclearin;} in the error rule's | |
4442 | action. | |
4443 | ||
4444 | For example, suppose that on a parse error, an error handling routine is | |
4445 | called that advances the input stream to some point where parsing should | |
4446 | once again commence. The next symbol returned by the lexical scanner is | |
4447 | probably correct. The previous look-ahead token ought to be discarded | |
4448 | with @samp{yyclearin;}. | |
4449 | ||
4450 | @vindex YYRECOVERING | |
4451 | The macro @code{YYRECOVERING} stands for an expression that has the | |
4452 | value 1 when the parser is recovering from a syntax error, and 0 the | |
4453 | rest of the time. A value of 1 indicates that error messages are | |
4454 | currently suppressed for new syntax errors. | |
4455 | ||
4456 | @node Context Dependency, Debugging, Error Recovery, Top | |
4457 | @chapter Handling Context Dependencies | |
4458 | ||
4459 | The Bison paradigm is to parse tokens first, then group them into larger | |
4460 | syntactic units. In many languages, the meaning of a token is affected by | |
4461 | its context. Although this violates the Bison paradigm, certain techniques | |
4462 | (known as @dfn{kludges}) may enable you to write Bison parsers for such | |
4463 | languages. | |
4464 | ||
4465 | @menu | |
4466 | * Semantic Tokens:: Token parsing can depend on the semantic context. | |
4467 | * Lexical Tie-ins:: Token parsing can depend on the syntactic context. | |
4468 | * Tie-in Recovery:: Lexical tie-ins have implications for how | |
4469 | error recovery rules must be written. | |
4470 | @end menu | |
4471 | ||
4472 | (Actually, ``kludge'' means any technique that gets its job done but is | |
4473 | neither clean nor robust.) | |
4474 | ||
4475 | @node Semantic Tokens, Lexical Tie-ins, , Context Dependency | |
4476 | @section Semantic Info in Token Types | |
4477 | ||
4478 | The C language has a context dependency: the way an identifier is used | |
4479 | depends on what its current meaning is. For example, consider this: | |
4480 | ||
4481 | @example | |
4482 | foo (x); | |
4483 | @end example | |
4484 | ||
4485 | This looks like a function call statement, but if @code{foo} is a typedef | |
4486 | name, then this is actually a declaration of @code{x}. How can a Bison | |
4487 | parser for C decide how to parse this input? | |
4488 | ||
4489 | The method used in GNU C is to have two different token types, | |
4490 | @code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an | |
4491 | identifier, it looks up the current declaration of the identifier in order | |
4492 | to decide which token type to return: @code{TYPENAME} if the identifier is | |
4493 | declared as a typedef, @code{IDENTIFIER} otherwise. | |
4494 | ||
4495 | The grammar rules can then express the context dependency by the choice of | |
4496 | token type to recognize. @code{IDENTIFIER} is accepted as an expression, | |
4497 | but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but | |
4498 | @code{IDENTIFIER} cannot. In contexts where the meaning of the identifier | |
4499 | is @emph{not} significant, such as in declarations that can shadow a | |
4500 | typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is | |
4501 | accepted---there is one rule for each of the two token types. | |
4502 | ||
4503 | This technique is simple to use if the decision of which kinds of | |
4504 | identifiers to allow is made at a place close to where the identifier is | |
4505 | parsed. But in C this is not always so: C allows a declaration to | |
4506 | redeclare a typedef name provided an explicit type has been specified | |
4507 | earlier: | |
4508 | ||
4509 | @example | |
4510 | typedef int foo, bar, lose; | |
4511 | static foo (bar); /* @r{redeclare @code{bar} as static variable} */ | |
4512 | static int foo (lose); /* @r{redeclare @code{foo} as function} */ | |
4513 | @end example | |
4514 | ||
4515 | Unfortunately, the name being declared is separated from the declaration | |
4516 | construct itself by a complicated syntactic structure---the ``declarator''. | |
4517 | ||
9ecbd125 | 4518 | As a result, part of the Bison parser for C needs to be duplicated, with |
14ded682 AD |
4519 | all the nonterminal names changed: once for parsing a declaration in |
4520 | which a typedef name can be redefined, and once for parsing a | |
4521 | declaration in which that can't be done. Here is a part of the | |
4522 | duplication, with actions omitted for brevity: | |
bfa74976 RS |
4523 | |
4524 | @example | |
4525 | initdcl: | |
4526 | declarator maybeasm '=' | |
4527 | init | |
4528 | | declarator maybeasm | |
4529 | ; | |
4530 | ||
4531 | notype_initdcl: | |
4532 | notype_declarator maybeasm '=' | |
4533 | init | |
4534 | | notype_declarator maybeasm | |
4535 | ; | |
4536 | @end example | |
4537 | ||
4538 | @noindent | |
4539 | Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl} | |
4540 | cannot. The distinction between @code{declarator} and | |
4541 | @code{notype_declarator} is the same sort of thing. | |
4542 | ||
4543 | There is some similarity between this technique and a lexical tie-in | |
4544 | (described next), in that information which alters the lexical analysis is | |
4545 | changed during parsing by other parts of the program. The difference is | |
4546 | here the information is global, and is used for other purposes in the | |
4547 | program. A true lexical tie-in has a special-purpose flag controlled by | |
4548 | the syntactic context. | |
4549 | ||
4550 | @node Lexical Tie-ins, Tie-in Recovery, Semantic Tokens, Context Dependency | |
4551 | @section Lexical Tie-ins | |
4552 | @cindex lexical tie-in | |
4553 | ||
4554 | One way to handle context-dependency is the @dfn{lexical tie-in}: a flag | |
4555 | which is set by Bison actions, whose purpose is to alter the way tokens are | |
4556 | parsed. | |
4557 | ||
4558 | For example, suppose we have a language vaguely like C, but with a special | |
4559 | construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes | |
4560 | an expression in parentheses in which all integers are hexadecimal. In | |
4561 | particular, the token @samp{a1b} must be treated as an integer rather than | |
4562 | as an identifier if it appears in that context. Here is how you can do it: | |
4563 | ||
4564 | @example | |
4565 | @group | |
4566 | %@{ | |
4567 | int hexflag; | |
4568 | %@} | |
4569 | %% | |
4570 | @dots{} | |
4571 | @end group | |
4572 | @group | |
4573 | expr: IDENTIFIER | |
4574 | | constant | |
4575 | | HEX '(' | |
4576 | @{ hexflag = 1; @} | |
4577 | expr ')' | |
4578 | @{ hexflag = 0; | |
4579 | $$ = $4; @} | |
4580 | | expr '+' expr | |
4581 | @{ $$ = make_sum ($1, $3); @} | |
4582 | @dots{} | |
4583 | ; | |
4584 | @end group | |
4585 | ||
4586 | @group | |
4587 | constant: | |
4588 | INTEGER | |
4589 | | STRING | |
4590 | ; | |
4591 | @end group | |
4592 | @end example | |
4593 | ||
4594 | @noindent | |
4595 | Here we assume that @code{yylex} looks at the value of @code{hexflag}; when | |
4596 | it is nonzero, all integers are parsed in hexadecimal, and tokens starting | |
4597 | with letters are parsed as integers if possible. | |
4598 | ||
75f5aaea MA |
4599 | The declaration of @code{hexflag} shown in the prologue of the parser file |
4600 | is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}). | |
4601 | You must also write the code in @code{yylex} to obey the flag. | |
bfa74976 RS |
4602 | |
4603 | @node Tie-in Recovery, , Lexical Tie-ins, Context Dependency | |
4604 | @section Lexical Tie-ins and Error Recovery | |
4605 | ||
4606 | Lexical tie-ins make strict demands on any error recovery rules you have. | |
4607 | @xref{Error Recovery}. | |
4608 | ||
4609 | The reason for this is that the purpose of an error recovery rule is to | |
4610 | abort the parsing of one construct and resume in some larger construct. | |
4611 | For example, in C-like languages, a typical error recovery rule is to skip | |
4612 | tokens until the next semicolon, and then start a new statement, like this: | |
4613 | ||
4614 | @example | |
4615 | stmt: expr ';' | |
4616 | | IF '(' expr ')' stmt @{ @dots{} @} | |
4617 | @dots{} | |
4618 | error ';' | |
4619 | @{ hexflag = 0; @} | |
4620 | ; | |
4621 | @end example | |
4622 | ||
4623 | If there is a syntax error in the middle of a @samp{hex (@var{expr})} | |
4624 | construct, this error rule will apply, and then the action for the | |
4625 | completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would | |
4626 | remain set for the entire rest of the input, or until the next @code{hex} | |
4627 | keyword, causing identifiers to be misinterpreted as integers. | |
4628 | ||
4629 | To avoid this problem the error recovery rule itself clears @code{hexflag}. | |
4630 | ||
4631 | There may also be an error recovery rule that works within expressions. | |
4632 | For example, there could be a rule which applies within parentheses | |
4633 | and skips to the close-parenthesis: | |
4634 | ||
4635 | @example | |
4636 | @group | |
4637 | expr: @dots{} | |
4638 | | '(' expr ')' | |
4639 | @{ $$ = $2; @} | |
4640 | | '(' error ')' | |
4641 | @dots{} | |
4642 | @end group | |
4643 | @end example | |
4644 | ||
4645 | If this rule acts within the @code{hex} construct, it is not going to abort | |
4646 | that construct (since it applies to an inner level of parentheses within | |
4647 | the construct). Therefore, it should not clear the flag: the rest of | |
4648 | the @code{hex} construct should be parsed with the flag still in effect. | |
4649 | ||
4650 | What if there is an error recovery rule which might abort out of the | |
4651 | @code{hex} construct or might not, depending on circumstances? There is no | |
4652 | way you can write the action to determine whether a @code{hex} construct is | |
4653 | being aborted or not. So if you are using a lexical tie-in, you had better | |
4654 | make sure your error recovery rules are not of this kind. Each rule must | |
4655 | be such that you can be sure that it always will, or always won't, have to | |
4656 | clear the flag. | |
4657 | ||
4658 | @node Debugging, Invocation, Context Dependency, Top | |
4659 | @chapter Debugging Your Parser | |
4660 | @findex YYDEBUG | |
4661 | @findex yydebug | |
4662 | @cindex debugging | |
4663 | @cindex tracing the parser | |
4664 | ||
4665 | If a Bison grammar compiles properly but doesn't do what you want when it | |
4666 | runs, the @code{yydebug} parser-trace feature can help you figure out why. | |
4667 | ||
4668 | To enable compilation of trace facilities, you must define the macro | |
75f5aaea MA |
4669 | @code{YYDEBUG} when you compile the parser. You could use @samp{-DYYDEBUG=1} |
4670 | as a compiler option or you could put @samp{#define YYDEBUG 1} in the prologue | |
4671 | of the grammar file (@pxref{Prologue, , The Prologue}). Alternatively, use the | |
4672 | @samp{-t} option when you run Bison (@pxref{Invocation, ,Invoking Bison}). | |
4673 | We always define @code{YYDEBUG} so that debugging is always possible. | |
bfa74976 | 4674 | |
75f5aaea MA |
4675 | The trace facility uses @code{stderr}, so you must add |
4676 | @w{@code{#include <stdio.h>}} to the prologue unless it is already there. | |
bfa74976 RS |
4677 | |
4678 | Once you have compiled the program with trace facilities, the way to | |
4679 | request a trace is to store a nonzero value in the variable @code{yydebug}. | |
4680 | You can do this by making the C code do it (in @code{main}, perhaps), or | |
4681 | you can alter the value with a C debugger. | |
4682 | ||
4683 | Each step taken by the parser when @code{yydebug} is nonzero produces a | |
4684 | line or two of trace information, written on @code{stderr}. The trace | |
4685 | messages tell you these things: | |
4686 | ||
4687 | @itemize @bullet | |
4688 | @item | |
4689 | Each time the parser calls @code{yylex}, what kind of token was read. | |
4690 | ||
4691 | @item | |
4692 | Each time a token is shifted, the depth and complete contents of the | |
4693 | state stack (@pxref{Parser States}). | |
4694 | ||
4695 | @item | |
4696 | Each time a rule is reduced, which rule it is, and the complete contents | |
4697 | of the state stack afterward. | |
4698 | @end itemize | |
4699 | ||
4700 | To make sense of this information, it helps to refer to the listing file | |
4701 | produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking Bison}). This file | |
4702 | shows the meaning of each state in terms of positions in various rules, and | |
4703 | also what each state will do with each possible input token. As you read | |
4704 | the successive trace messages, you can see that the parser is functioning | |
4705 | according to its specification in the listing file. Eventually you will | |
4706 | arrive at the place where something undesirable happens, and you will see | |
4707 | which parts of the grammar are to blame. | |
4708 | ||
4709 | The parser file is a C program and you can use C debuggers on it, but it's | |
4710 | not easy to interpret what it is doing. The parser function is a | |
4711 | finite-state machine interpreter, and aside from the actions it executes | |
4712 | the same code over and over. Only the values of variables show where in | |
4713 | the grammar it is working. | |
4714 | ||
4715 | @findex YYPRINT | |
4716 | The debugging information normally gives the token type of each token | |
4717 | read, but not its semantic value. You can optionally define a macro | |
4718 | named @code{YYPRINT} to provide a way to print the value. If you define | |
4719 | @code{YYPRINT}, it should take three arguments. The parser will pass a | |
4720 | standard I/O stream, the numeric code for the token type, and the token | |
4721 | value (from @code{yylval}). | |
4722 | ||
4723 | Here is an example of @code{YYPRINT} suitable for the multi-function | |
4724 | calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}): | |
4725 | ||
4726 | @smallexample | |
4727 | #define YYPRINT(file, type, value) yyprint (file, type, value) | |
4728 | ||
4729 | static void | |
13863333 | 4730 | yyprint (FILE *file, int type, YYSTYPE value) |
bfa74976 RS |
4731 | @{ |
4732 | if (type == VAR) | |
4733 | fprintf (file, " %s", value.tptr->name); | |
4734 | else if (type == NUM) | |
4735 | fprintf (file, " %d", value.val); | |
4736 | @} | |
4737 | @end smallexample | |
4738 | ||
4739 | @node Invocation, Table of Symbols, Debugging, Top | |
4740 | @chapter Invoking Bison | |
4741 | @cindex invoking Bison | |
4742 | @cindex Bison invocation | |
4743 | @cindex options for invoking Bison | |
4744 | ||
4745 | The usual way to invoke Bison is as follows: | |
4746 | ||
4747 | @example | |
4748 | bison @var{infile} | |
4749 | @end example | |
4750 | ||
4751 | Here @var{infile} is the grammar file name, which usually ends in | |
4752 | @samp{.y}. The parser file's name is made by replacing the @samp{.y} | |
4753 | with @samp{.tab.c}. Thus, the @samp{bison foo.y} filename yields | |
4754 | @file{foo.tab.c}, and the @samp{bison hack/foo.y} filename yields | |
234a3be3 | 4755 | @file{hack/foo.tab.c}. It's is also possible, in case you are writting |
79282c6c | 4756 | C++ code instead of C in your grammar file, to name it @file{foo.ypp} |
234a3be3 AD |
4757 | or @file{foo.y++}. Then, the output files will take an extention like |
4758 | the given one as input (repectively @file{foo.tab.cpp} and @file{foo.tab.c++}). | |
4759 | This feature takes effect with all options that manipulate filenames like | |
4760 | @samp{-o} or @samp{-d}. | |
4761 | ||
4762 | For example : | |
4763 | ||
4764 | @example | |
4765 | bison -d @var{infile.yxx} | |
4766 | @end example | |
84163231 | 4767 | @noindent |
234a3be3 AD |
4768 | will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}. and |
4769 | ||
4770 | @example | |
4771 | bison -d @var{infile.y} -o @var{output.c++} | |
4772 | @end example | |
84163231 | 4773 | @noindent |
234a3be3 AD |
4774 | will produce @file{output.c++} and @file{outfile.h++}. |
4775 | ||
bfa74976 RS |
4776 | |
4777 | @menu | |
13863333 | 4778 | * Bison Options:: All the options described in detail, |
bfa74976 | 4779 | in alphabetical order by short options. |
9ecbd125 | 4780 | * Environment Variables:: Variables which affect Bison execution. |
bfa74976 RS |
4781 | * Option Cross Key:: Alphabetical list of long options. |
4782 | * VMS Invocation:: Bison command syntax on VMS. | |
4783 | @end menu | |
4784 | ||
9ecbd125 | 4785 | @node Bison Options, Environment Variables, , Invocation |
bfa74976 RS |
4786 | @section Bison Options |
4787 | ||
4788 | Bison supports both traditional single-letter options and mnemonic long | |
4789 | option names. Long option names are indicated with @samp{--} instead of | |
4790 | @samp{-}. Abbreviations for option names are allowed as long as they | |
4791 | are unique. When a long option takes an argument, like | |
4792 | @samp{--file-prefix}, connect the option name and the argument with | |
4793 | @samp{=}. | |
4794 | ||
4795 | Here is a list of options that can be used with Bison, alphabetized by | |
4796 | short option. It is followed by a cross key alphabetized by long | |
4797 | option. | |
4798 | ||
89cab50d AD |
4799 | @c Please, keep this ordered as in `bison --help'. |
4800 | @noindent | |
4801 | Operations modes: | |
4802 | @table @option | |
4803 | @item -h | |
4804 | @itemx --help | |
4805 | Print a summary of the command-line options to Bison and exit. | |
bfa74976 | 4806 | |
89cab50d AD |
4807 | @item -V |
4808 | @itemx --version | |
4809 | Print the version number of Bison and exit. | |
bfa74976 | 4810 | |
89cab50d AD |
4811 | @need 1750 |
4812 | @item -y | |
4813 | @itemx --yacc | |
4814 | @itemx --fixed-output-files | |
4815 | Equivalent to @samp{-o y.tab.c}; the parser output file is called | |
4816 | @file{y.tab.c}, and the other outputs are called @file{y.output} and | |
4817 | @file{y.tab.h}. The purpose of this option is to imitate Yacc's output | |
4818 | file name conventions. Thus, the following shell script can substitute | |
4819 | for Yacc:@refill | |
bfa74976 | 4820 | |
89cab50d AD |
4821 | @example |
4822 | bison -y $* | |
4823 | @end example | |
4824 | @end table | |
4825 | ||
4826 | @noindent | |
4827 | Tuning the parser: | |
4828 | ||
4829 | @table @option | |
cd5bd6ac AD |
4830 | @item -S @var{file} |
4831 | @itemx --skeleton=@var{file} | |
4832 | Specify the skeleton to use. You probably don't need this option unless | |
4833 | you are developing Bison. | |
4834 | ||
89cab50d AD |
4835 | @item -t |
4836 | @itemx --debug | |
6deb4447 AD |
4837 | Output a definition of the macro @code{YYDEBUG} into the parser file, so |
4838 | that the debugging facilities are compiled. @xref{Debugging, ,Debugging | |
4839 | Your Parser}. | |
89cab50d AD |
4840 | |
4841 | @item --locations | |
4842 | Pretend that @code{%locactions} was specified. @xref{Decl Summary}. | |
4843 | ||
4844 | @item -p @var{prefix} | |
4845 | @itemx --name-prefix=@var{prefix} | |
4846 | Rename the external symbols used in the parser so that they start with | |
4847 | @var{prefix} instead of @samp{yy}. The precise list of symbols renamed | |
4848 | is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs}, | |
4849 | @code{yylval}, @code{yychar} and @code{yydebug}. | |
4850 | ||
4851 | For example, if you use @samp{-p c}, the names become @code{cparse}, | |
4852 | @code{clex}, and so on. | |
4853 | ||
4854 | @xref{Multiple Parsers, ,Multiple Parsers in the Same Program}. | |
bfa74976 RS |
4855 | |
4856 | @item -l | |
4857 | @itemx --no-lines | |
4858 | Don't put any @code{#line} preprocessor commands in the parser file. | |
4859 | Ordinarily Bison puts them in the parser file so that the C compiler | |
4860 | and debuggers will associate errors with your source file, the | |
4861 | grammar file. This option causes them to associate errors with the | |
95e742f7 | 4862 | parser file, treating it as an independent source file in its own right. |
bfa74976 | 4863 | |
931c7513 RS |
4864 | @item -n |
4865 | @itemx --no-parser | |
6deb4447 | 4866 | Pretend that @code{%no_parser} was specified. @xref{Decl Summary}. |
931c7513 | 4867 | |
89cab50d AD |
4868 | @item -k |
4869 | @itemx --token-table | |
4870 | Pretend that @code{%token_table} was specified. @xref{Decl Summary}. | |
4871 | @end table | |
bfa74976 | 4872 | |
89cab50d AD |
4873 | @noindent |
4874 | Adjust the output: | |
bfa74976 | 4875 | |
89cab50d AD |
4876 | @table @option |
4877 | @item -d | |
4878 | @itemx --defines | |
6deb4447 AD |
4879 | Pretend that @code{%verbose} was specified, i.e., write an extra output |
4880 | file containing macro definitions for the token type names defined in | |
4881 | the grammar and the semantic value type @code{YYSTYPE}, as well as a few | |
4882 | @code{extern} variable declarations. @xref{Decl Summary}. | |
931c7513 | 4883 | |
89cab50d AD |
4884 | @item -b @var{file-prefix} |
4885 | @itemx --file-prefix=@var{prefix} | |
4886 | Specify a prefix to use for all Bison output file names. The names are | |
4887 | chosen as if the input file were named @file{@var{prefix}.c}. | |
bfa74976 RS |
4888 | |
4889 | @item -v | |
4890 | @itemx --verbose | |
6deb4447 AD |
4891 | Pretend that @code{%verbose} was specified, i.e, write an extra output |
4892 | file containing verbose descriptions of the grammar and | |
4893 | parser. @xref{Decl Summary}, for more. | |
bfa74976 | 4894 | |
89cab50d AD |
4895 | @item -o @var{outfile} |
4896 | @itemx --output-file=@var{outfile} | |
4897 | Specify the name @var{outfile} for the parser file. | |
bfa74976 | 4898 | |
89cab50d AD |
4899 | The other output files' names are constructed from @var{outfile} |
4900 | as described under the @samp{-v} and @samp{-d} options. | |
bfa74976 RS |
4901 | @end table |
4902 | ||
9ecbd125 JT |
4903 | @node Environment Variables, Option Cross Key, Bison Options, Invocation |
4904 | @section Environment Variables | |
4905 | @cindex environment variables | |
4906 | @cindex BISON_HAIRY | |
4907 | @cindex BISON_SIMPLE | |
4908 | ||
4909 | Here is a list of environment variables which affect the way Bison | |
4910 | runs. | |
4911 | ||
4912 | @table @samp | |
4913 | @item BISON_SIMPLE | |
4914 | @itemx BISON_HAIRY | |
4915 | Much of the parser generated by Bison is copied verbatim from a file | |
4916 | called @file{bison.simple}. If Bison cannot find that file, or if you | |
4917 | would like to direct Bison to use a different copy, setting the | |
4918 | environment variable @code{BISON_SIMPLE} to the path of the file will | |
4919 | cause Bison to use that copy instead. | |
4920 | ||
13863333 | 4921 | When the @samp{%semantic_parser} declaration is used, Bison copies from |
9ecbd125 JT |
4922 | a file called @file{bison.hairy} instead. The location of this file can |
4923 | also be specified or overridden in a similar fashion, with the | |
4924 | @code{BISON_HAIRY} environment variable. | |
4925 | ||
4926 | @end table | |
4927 | ||
4928 | @node Option Cross Key, VMS Invocation, Environment Variables, Invocation | |
bfa74976 RS |
4929 | @section Option Cross Key |
4930 | ||
4931 | Here is a list of options, alphabetized by long option, to help you find | |
4932 | the corresponding short option. | |
4933 | ||
4934 | @tex | |
4935 | \def\leaderfill{\leaders\hbox to 1em{\hss.\hss}\hfill} | |
4936 | ||
4937 | {\tt | |
4938 | \line{ --debug \leaderfill -t} | |
4939 | \line{ --defines \leaderfill -d} | |
4940 | \line{ --file-prefix \leaderfill -b} | |
4941 | \line{ --fixed-output-files \leaderfill -y} | |
ff51d159 | 4942 | \line{ --help \leaderfill -h} |
bfa74976 RS |
4943 | \line{ --name-prefix \leaderfill -p} |
4944 | \line{ --no-lines \leaderfill -l} | |
931c7513 | 4945 | \line{ --no-parser \leaderfill -n} |
bfa74976 | 4946 | \line{ --output-file \leaderfill -o} |
931c7513 | 4947 | \line{ --token-table \leaderfill -k} |
bfa74976 RS |
4948 | \line{ --verbose \leaderfill -v} |
4949 | \line{ --version \leaderfill -V} | |
4950 | \line{ --yacc \leaderfill -y} | |
4951 | } | |
4952 | @end tex | |
4953 | ||
4954 | @ifinfo | |
4955 | @example | |
4956 | --debug -t | |
4957 | --defines -d | |
4958 | --file-prefix=@var{prefix} -b @var{file-prefix} | |
4959 | --fixed-output-files --yacc -y | |
ff51d159 | 4960 | --help -h |
931c7513 | 4961 | --name-prefix=@var{prefix} -p @var{name-prefix} |
bfa74976 | 4962 | --no-lines -l |
931c7513 | 4963 | --no-parser -n |
bfa74976 | 4964 | --output-file=@var{outfile} -o @var{outfile} |
931c7513 | 4965 | --token-table -k |
bfa74976 RS |
4966 | --verbose -v |
4967 | --version -V | |
4968 | @end example | |
4969 | @end ifinfo | |
4970 | ||
4971 | @node VMS Invocation, , Option Cross Key, Invocation | |
4972 | @section Invoking Bison under VMS | |
4973 | @cindex invoking Bison under VMS | |
4974 | @cindex VMS | |
4975 | ||
4976 | The command line syntax for Bison on VMS is a variant of the usual | |
4977 | Bison command syntax---adapted to fit VMS conventions. | |
4978 | ||
4979 | To find the VMS equivalent for any Bison option, start with the long | |
4980 | option, and substitute a @samp{/} for the leading @samp{--}, and | |
4981 | substitute a @samp{_} for each @samp{-} in the name of the long option. | |
4982 | For example, the following invocation under VMS: | |
4983 | ||
4984 | @example | |
4985 | bison /debug/name_prefix=bar foo.y | |
4986 | @end example | |
4987 | ||
4988 | @noindent | |
4989 | is equivalent to the following command under POSIX. | |
4990 | ||
4991 | @example | |
4992 | bison --debug --name-prefix=bar foo.y | |
4993 | @end example | |
4994 | ||
4995 | The VMS file system does not permit filenames such as | |
4996 | @file{foo.tab.c}. In the above example, the output file | |
4997 | would instead be named @file{foo_tab.c}. | |
4998 | ||
4999 | @node Table of Symbols, Glossary, Invocation, Top | |
5000 | @appendix Bison Symbols | |
5001 | @cindex Bison symbols, table of | |
5002 | @cindex symbols in Bison, table of | |
5003 | ||
5004 | @table @code | |
5005 | @item error | |
5006 | A token name reserved for error recovery. This token may be used in | |
5007 | grammar rules so as to allow the Bison parser to recognize an error in | |
5008 | the grammar without halting the process. In effect, a sentence | |
5009 | containing an error may be recognized as valid. On a parse error, the | |
5010 | token @code{error} becomes the current look-ahead token. Actions | |
5011 | corresponding to @code{error} are then executed, and the look-ahead | |
5012 | token is reset to the token that originally caused the violation. | |
5013 | @xref{Error Recovery}. | |
5014 | ||
5015 | @item YYABORT | |
5016 | Macro to pretend that an unrecoverable syntax error has occurred, by | |
5017 | making @code{yyparse} return 1 immediately. The error reporting | |
ceed8467 AD |
5018 | function @code{yyerror} is not called. @xref{Parser Function, ,The |
5019 | Parser Function @code{yyparse}}. | |
bfa74976 RS |
5020 | |
5021 | @item YYACCEPT | |
5022 | Macro to pretend that a complete utterance of the language has been | |
13863333 | 5023 | read, by making @code{yyparse} return 0 immediately. |
bfa74976 RS |
5024 | @xref{Parser Function, ,The Parser Function @code{yyparse}}. |
5025 | ||
5026 | @item YYBACKUP | |
5027 | Macro to discard a value from the parser stack and fake a look-ahead | |
5028 | token. @xref{Action Features, ,Special Features for Use in Actions}. | |
5029 | ||
5030 | @item YYERROR | |
5031 | Macro to pretend that a syntax error has just been detected: call | |
5032 | @code{yyerror} and then perform normal error recovery if possible | |
5033 | (@pxref{Error Recovery}), or (if recovery is impossible) make | |
5034 | @code{yyparse} return 1. @xref{Error Recovery}. | |
5035 | ||
5036 | @item YYERROR_VERBOSE | |
5037 | Macro that you define with @code{#define} in the Bison declarations | |
5038 | section to request verbose, specific error message strings when | |
5039 | @code{yyerror} is called. | |
5040 | ||
5041 | @item YYINITDEPTH | |
5042 | Macro for specifying the initial size of the parser stack. | |
5043 | @xref{Stack Overflow}. | |
5044 | ||
c656404a RS |
5045 | @item YYLEX_PARAM |
5046 | Macro for specifying an extra argument (or list of extra arguments) for | |
5047 | @code{yyparse} to pass to @code{yylex}. @xref{Pure Calling,, Calling | |
5048 | Conventions for Pure Parsers}. | |
5049 | ||
bfa74976 RS |
5050 | @item YYLTYPE |
5051 | Macro for the data type of @code{yylloc}; a structure with four | |
847bf1f5 | 5052 | members. @xref{Location Type, , Data Types of Locations}. |
bfa74976 | 5053 | |
931c7513 RS |
5054 | @item yyltype |
5055 | Default value for YYLTYPE. | |
5056 | ||
bfa74976 RS |
5057 | @item YYMAXDEPTH |
5058 | Macro for specifying the maximum size of the parser stack. | |
5059 | @xref{Stack Overflow}. | |
5060 | ||
c656404a RS |
5061 | @item YYPARSE_PARAM |
5062 | Macro for specifying the name of a parameter that @code{yyparse} should | |
5063 | accept. @xref{Pure Calling,, Calling Conventions for Pure Parsers}. | |
5064 | ||
bfa74976 RS |
5065 | @item YYRECOVERING |
5066 | Macro whose value indicates whether the parser is recovering from a | |
5067 | syntax error. @xref{Action Features, ,Special Features for Use in Actions}. | |
5068 | ||
f9a8293a AD |
5069 | @item YYSTACK_USE_ALLOCA |
5070 | Macro used to control the use of @code{alloca}. If defined to @samp{0}, | |
5071 | the parser will not use @code{alloca} but @code{malloc} when trying to | |
5072 | grow its internal stacks. Do @emph{not} define @code{YYSTACK_USE_ALLOCA} | |
5073 | to anything else. | |
5074 | ||
bfa74976 RS |
5075 | @item YYSTYPE |
5076 | Macro for the data type of semantic values; @code{int} by default. | |
5077 | @xref{Value Type, ,Data Types of Semantic Values}. | |
5078 | ||
5079 | @item yychar | |
13863333 AD |
5080 | External integer variable that contains the integer value of the current |
5081 | look-ahead token. (In a pure parser, it is a local variable within | |
5082 | @code{yyparse}.) Error-recovery rule actions may examine this variable. | |
5083 | @xref{Action Features, ,Special Features for Use in Actions}. | |
bfa74976 RS |
5084 | |
5085 | @item yyclearin | |
5086 | Macro used in error-recovery rule actions. It clears the previous | |
5087 | look-ahead token. @xref{Error Recovery}. | |
5088 | ||
5089 | @item yydebug | |
5090 | External integer variable set to zero by default. If @code{yydebug} | |
5091 | is given a nonzero value, the parser will output information on input | |
5092 | symbols and parser action. @xref{Debugging, ,Debugging Your Parser}. | |
5093 | ||
5094 | @item yyerrok | |
5095 | Macro to cause parser to recover immediately to its normal mode | |
5096 | after a parse error. @xref{Error Recovery}. | |
5097 | ||
5098 | @item yyerror | |
5099 | User-supplied function to be called by @code{yyparse} on error. The | |
5100 | function receives one argument, a pointer to a character string | |
13863333 AD |
5101 | containing an error message. @xref{Error Reporting, ,The Error |
5102 | Reporting Function @code{yyerror}}. | |
bfa74976 RS |
5103 | |
5104 | @item yylex | |
5105 | User-supplied lexical analyzer function, called with no arguments | |
5106 | to get the next token. @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}. | |
5107 | ||
5108 | @item yylval | |
5109 | External variable in which @code{yylex} should place the semantic | |
5110 | value associated with a token. (In a pure parser, it is a local | |
5111 | variable within @code{yyparse}, and its address is passed to | |
5112 | @code{yylex}.) @xref{Token Values, ,Semantic Values of Tokens}. | |
5113 | ||
5114 | @item yylloc | |
13863333 AD |
5115 | External variable in which @code{yylex} should place the line and column |
5116 | numbers associated with a token. (In a pure parser, it is a local | |
5117 | variable within @code{yyparse}, and its address is passed to | |
bfa74976 | 5118 | @code{yylex}.) You can ignore this variable if you don't use the |
13863333 AD |
5119 | @samp{@@} feature in the grammar actions. @xref{Token Positions, |
5120 | ,Textual Positions of Tokens}. | |
bfa74976 RS |
5121 | |
5122 | @item yynerrs | |
13863333 AD |
5123 | Global variable which Bison increments each time there is a parse error. |
5124 | (In a pure parser, it is a local variable within @code{yyparse}.) | |
5125 | @xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}. | |
bfa74976 RS |
5126 | |
5127 | @item yyparse | |
5128 | The parser function produced by Bison; call this function to start | |
5129 | parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}. | |
5130 | ||
6deb4447 AD |
5131 | @item %debug |
5132 | Equip the parser for debugging. @xref{Decl Summary}. | |
5133 | ||
5134 | @item %defines | |
5135 | Bison declaration to create a header file meant for the scanner. | |
5136 | @xref{Decl Summary}. | |
5137 | ||
f9a8293a AD |
5138 | @c @item %source_extension |
5139 | @c Bison declaration to specify the generated parser output file extension. | |
5140 | @c @xref{Decl Summary}. | |
5141 | @c | |
5142 | @c @item %header_extension | |
5143 | @c Bison declaration to specify the generated parser header file extension | |
5144 | @c if required. @xref{Decl Summary}. | |
5145 | ||
bfa74976 RS |
5146 | @item %left |
5147 | Bison declaration to assign left associativity to token(s). | |
5148 | @xref{Precedence Decl, ,Operator Precedence}. | |
5149 | ||
931c7513 RS |
5150 | @item %no_lines |
5151 | Bison declaration to avoid generating @code{#line} directives in the | |
5152 | parser file. @xref{Decl Summary}. | |
5153 | ||
bfa74976 | 5154 | @item %nonassoc |
14ded682 | 5155 | Bison declaration to assign non-associativity to token(s). |
bfa74976 RS |
5156 | @xref{Precedence Decl, ,Operator Precedence}. |
5157 | ||
5158 | @item %prec | |
5159 | Bison declaration to assign a precedence to a specific rule. | |
5160 | @xref{Contextual Precedence, ,Context-Dependent Precedence}. | |
5161 | ||
5162 | @item %pure_parser | |
5163 | Bison declaration to request a pure (reentrant) parser. | |
5164 | @xref{Pure Decl, ,A Pure (Reentrant) Parser}. | |
5165 | ||
5166 | @item %right | |
5167 | Bison declaration to assign right associativity to token(s). | |
5168 | @xref{Precedence Decl, ,Operator Precedence}. | |
5169 | ||
5170 | @item %start | |
5171 | Bison declaration to specify the start symbol. @xref{Start Decl, ,The Start-Symbol}. | |
5172 | ||
5173 | @item %token | |
5174 | Bison declaration to declare token(s) without specifying precedence. | |
5175 | @xref{Token Decl, ,Token Type Names}. | |
5176 | ||
931c7513 RS |
5177 | @item %token_table |
5178 | Bison declaration to include a token name table in the parser file. | |
5179 | @xref{Decl Summary}. | |
5180 | ||
bfa74976 RS |
5181 | @item %type |
5182 | Bison declaration to declare nonterminals. @xref{Type Decl, ,Nonterminal Symbols}. | |
5183 | ||
5184 | @item %union | |
5185 | Bison declaration to specify several possible data types for semantic | |
5186 | values. @xref{Union Decl, ,The Collection of Value Types}. | |
5187 | @end table | |
5188 | ||
5189 | These are the punctuation and delimiters used in Bison input: | |
5190 | ||
5191 | @table @samp | |
5192 | @item %% | |
5193 | Delimiter used to separate the grammar rule section from the | |
75f5aaea | 5194 | Bison declarations section or the epilogue. |
bfa74976 RS |
5195 | @xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}. |
5196 | ||
5197 | @item %@{ %@} | |
89cab50d | 5198 | All code listed between @samp{%@{} and @samp{%@}} is copied directly to |
75f5aaea MA |
5199 | the output file uninterpreted. Such code forms the prologue of the input |
5200 | file. @xref{Grammar Outline, ,Outline of a Bison | |
89cab50d | 5201 | Grammar}. |
bfa74976 RS |
5202 | |
5203 | @item /*@dots{}*/ | |
5204 | Comment delimiters, as in C. | |
5205 | ||
5206 | @item : | |
89cab50d AD |
5207 | Separates a rule's result from its components. @xref{Rules, ,Syntax of |
5208 | Grammar Rules}. | |
bfa74976 RS |
5209 | |
5210 | @item ; | |
5211 | Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}. | |
5212 | ||
5213 | @item | | |
5214 | Separates alternate rules for the same result nonterminal. | |
5215 | @xref{Rules, ,Syntax of Grammar Rules}. | |
5216 | @end table | |
5217 | ||
f2b5126e | 5218 | @node Glossary, Copying This Manual, Table of Symbols, Top |
bfa74976 RS |
5219 | @appendix Glossary |
5220 | @cindex glossary | |
5221 | ||
5222 | @table @asis | |
5223 | @item Backus-Naur Form (BNF) | |
5224 | Formal method of specifying context-free grammars. BNF was first used | |
89cab50d AD |
5225 | in the @cite{ALGOL-60} report, 1963. @xref{Language and Grammar, |
5226 | ,Languages and Context-Free Grammars}. | |
bfa74976 RS |
5227 | |
5228 | @item Context-free grammars | |
5229 | Grammars specified as rules that can be applied regardless of context. | |
5230 | Thus, if there is a rule which says that an integer can be used as an | |
5231 | expression, integers are allowed @emph{anywhere} an expression is | |
89cab50d AD |
5232 | permitted. @xref{Language and Grammar, ,Languages and Context-Free |
5233 | Grammars}. | |
bfa74976 RS |
5234 | |
5235 | @item Dynamic allocation | |
5236 | Allocation of memory that occurs during execution, rather than at | |
5237 | compile time or on entry to a function. | |
5238 | ||
5239 | @item Empty string | |
5240 | Analogous to the empty set in set theory, the empty string is a | |
5241 | character string of length zero. | |
5242 | ||
5243 | @item Finite-state stack machine | |
5244 | A ``machine'' that has discrete states in which it is said to exist at | |
5245 | each instant in time. As input to the machine is processed, the | |
5246 | machine moves from state to state as specified by the logic of the | |
5247 | machine. In the case of the parser, the input is the language being | |
5248 | parsed, and the states correspond to various stages in the grammar | |
5249 | rules. @xref{Algorithm, ,The Bison Parser Algorithm }. | |
5250 | ||
5251 | @item Grouping | |
5252 | A language construct that is (in general) grammatically divisible; | |
13863333 | 5253 | for example, `expression' or `declaration' in C. |
bfa74976 RS |
5254 | @xref{Language and Grammar, ,Languages and Context-Free Grammars}. |
5255 | ||
5256 | @item Infix operator | |
5257 | An arithmetic operator that is placed between the operands on which it | |
5258 | performs some operation. | |
5259 | ||
5260 | @item Input stream | |
5261 | A continuous flow of data between devices or programs. | |
5262 | ||
5263 | @item Language construct | |
5264 | One of the typical usage schemas of the language. For example, one of | |
5265 | the constructs of the C language is the @code{if} statement. | |
5266 | @xref{Language and Grammar, ,Languages and Context-Free Grammars}. | |
5267 | ||
5268 | @item Left associativity | |
5269 | Operators having left associativity are analyzed from left to right: | |
5270 | @samp{a+b+c} first computes @samp{a+b} and then combines with | |
5271 | @samp{c}. @xref{Precedence, ,Operator Precedence}. | |
5272 | ||
5273 | @item Left recursion | |
89cab50d AD |
5274 | A rule whose result symbol is also its first component symbol; for |
5275 | example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive | |
5276 | Rules}. | |
bfa74976 RS |
5277 | |
5278 | @item Left-to-right parsing | |
5279 | Parsing a sentence of a language by analyzing it token by token from | |
5280 | left to right. @xref{Algorithm, ,The Bison Parser Algorithm }. | |
5281 | ||
5282 | @item Lexical analyzer (scanner) | |
5283 | A function that reads an input stream and returns tokens one by one. | |
5284 | @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}. | |
5285 | ||
5286 | @item Lexical tie-in | |
5287 | A flag, set by actions in the grammar rules, which alters the way | |
5288 | tokens are parsed. @xref{Lexical Tie-ins}. | |
5289 | ||
931c7513 | 5290 | @item Literal string token |
14ded682 | 5291 | A token which consists of two or more fixed characters. @xref{Symbols}. |
931c7513 | 5292 | |
bfa74976 | 5293 | @item Look-ahead token |
89cab50d AD |
5294 | A token already read but not yet shifted. @xref{Look-Ahead, ,Look-Ahead |
5295 | Tokens}. | |
bfa74976 RS |
5296 | |
5297 | @item LALR(1) | |
5298 | The class of context-free grammars that Bison (like most other parser | |
5299 | generators) can handle; a subset of LR(1). @xref{Mystery Conflicts, , | |
5300 | Mysterious Reduce/Reduce Conflicts}. | |
5301 | ||
5302 | @item LR(1) | |
5303 | The class of context-free grammars in which at most one token of | |
5304 | look-ahead is needed to disambiguate the parsing of any piece of input. | |
5305 | ||
5306 | @item Nonterminal symbol | |
5307 | A grammar symbol standing for a grammatical construct that can | |
5308 | be expressed through rules in terms of smaller constructs; in other | |
5309 | words, a construct that is not a token. @xref{Symbols}. | |
5310 | ||
5311 | @item Parse error | |
5312 | An error encountered during parsing of an input stream due to invalid | |
5313 | syntax. @xref{Error Recovery}. | |
5314 | ||
5315 | @item Parser | |
5316 | A function that recognizes valid sentences of a language by analyzing | |
5317 | the syntax structure of a set of tokens passed to it from a lexical | |
5318 | analyzer. | |
5319 | ||
5320 | @item Postfix operator | |
5321 | An arithmetic operator that is placed after the operands upon which it | |
5322 | performs some operation. | |
5323 | ||
5324 | @item Reduction | |
5325 | Replacing a string of nonterminals and/or terminals with a single | |
89cab50d AD |
5326 | nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison |
5327 | Parser Algorithm }. | |
bfa74976 RS |
5328 | |
5329 | @item Reentrant | |
5330 | A reentrant subprogram is a subprogram which can be in invoked any | |
5331 | number of times in parallel, without interference between the various | |
5332 | invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}. | |
5333 | ||
5334 | @item Reverse polish notation | |
5335 | A language in which all operators are postfix operators. | |
5336 | ||
5337 | @item Right recursion | |
89cab50d AD |
5338 | A rule whose result symbol is also its last component symbol; for |
5339 | example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive | |
5340 | Rules}. | |
bfa74976 RS |
5341 | |
5342 | @item Semantics | |
5343 | In computer languages, the semantics are specified by the actions | |
5344 | taken for each instance of the language, i.e., the meaning of | |
5345 | each statement. @xref{Semantics, ,Defining Language Semantics}. | |
5346 | ||
5347 | @item Shift | |
5348 | A parser is said to shift when it makes the choice of analyzing | |
5349 | further input from the stream rather than reducing immediately some | |
5350 | already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm }. | |
5351 | ||
5352 | @item Single-character literal | |
5353 | A single character that is recognized and interpreted as is. | |
5354 | @xref{Grammar in Bison, ,From Formal Rules to Bison Input}. | |
5355 | ||
5356 | @item Start symbol | |
5357 | The nonterminal symbol that stands for a complete valid utterance in | |
5358 | the language being parsed. The start symbol is usually listed as the | |
13863333 | 5359 | first nonterminal symbol in a language specification. |
bfa74976 RS |
5360 | @xref{Start Decl, ,The Start-Symbol}. |
5361 | ||
5362 | @item Symbol table | |
5363 | A data structure where symbol names and associated data are stored | |
5364 | during parsing to allow for recognition and use of existing | |
5365 | information in repeated uses of a symbol. @xref{Multi-function Calc}. | |
5366 | ||
5367 | @item Token | |
5368 | A basic, grammatically indivisible unit of a language. The symbol | |
5369 | that describes a token in the grammar is a terminal symbol. | |
5370 | The input of the Bison parser is a stream of tokens which comes from | |
5371 | the lexical analyzer. @xref{Symbols}. | |
5372 | ||
5373 | @item Terminal symbol | |
89cab50d AD |
5374 | A grammar symbol that has no rules in the grammar and therefore is |
5375 | grammatically indivisible. The piece of text it represents is a token. | |
5376 | @xref{Language and Grammar, ,Languages and Context-Free Grammars}. | |
bfa74976 RS |
5377 | @end table |
5378 | ||
f2b5126e PB |
5379 | @node Copying This Manual, Index, Glossary, Top |
5380 | @appendix Copying This Manual | |
f9a8293a | 5381 | |
f2b5126e PB |
5382 | @menu |
5383 | * GNU Free Documentation License:: License for copying this manual. | |
5384 | @end menu | |
f9a8293a | 5385 | |
f2b5126e PB |
5386 | @include fdl.texi |
5387 | ||
5388 | @node Index, , Copying This Manual, Top | |
bfa74976 RS |
5389 | @unnumbered Index |
5390 | ||
5391 | @printindex cp | |
5392 | ||
bfa74976 | 5393 | @bye |