* Precedence Only:: How to specify precedence only.
* Precedence Examples:: How these features are used in the previous example.
* How Precedence:: How they work.
+* Non Operators:: Using precedence for general conflicts.
Tuning LR
term:
'(' expr ')'
| term '!'
-| NUMBER
+| "number"
;
@end group
@end example
@example
@group
if_stmt:
- IF expr THEN stmt
-| IF expr THEN stmt ELSE stmt
+ "if" expr "then" stmt
+| "if" expr "then" stmt "else" stmt
;
@end group
@end example
@noindent
-Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
-terminal symbols for specific keyword tokens.
+Here @code{"if"}, @code{"then"} and @code{"else"} are terminal symbols for
+specific keyword tokens.
-When the @code{ELSE} token is read and becomes the lookahead token, the
+When the @code{"else"} token is read and becomes the lookahead token, the
contents of the stack (assuming the input is valid) are just right for
reduction by the first rule. But it is also legitimate to shift the
-@code{ELSE}, because that would lead to eventual reduction by the second
+@code{"else"}, because that would lead to eventual reduction by the second
rule.
This situation, where either a shift or a reduction would be valid, is
operator precedence declarations. To see the reason for this, let's
contrast it with the other alternative.
-Since the parser prefers to shift the @code{ELSE}, the result is to attach
+Since the parser prefers to shift the @code{"else"}, the result is to attach
the else-clause to the innermost if-statement, making these two inputs
equivalent:
@example
-if x then if y then win (); else lose;
+if x then if y then win; else lose;
-if x then do; if y then win (); else lose; end;
+if x then do; if y then win; else lose; end;
@end example
But if the parser chose to reduce when possible rather than shift, the
making these two inputs equivalent:
@example
-if x then if y then win (); else lose;
+if x then if y then win; else lose;
-if x then do; if y then win (); end; else lose;
+if x then do; if y then win; end; else lose;
@end example
The conflict exists because the grammar as written is ambiguous: either
Algol 60 and is called the ``dangling @code{else}'' ambiguity.
To avoid warnings from Bison about predictable, legitimate shift/reduce
-conflicts, use the @code{%expect @var{n}} declaration.
+conflicts, you can use the @code{%expect @var{n}} declaration.
There will be no warning as long as the number of shift/reduce conflicts
is exactly @var{n}, and Bison will report an error if there is a
different number.
-@xref{Expect Decl, ,Suppressing Conflict Warnings}.
+@xref{Expect Decl, ,Suppressing Conflict Warnings}. However, we don't
+recommend the use of @code{%expect} (except @samp{%expect 0}!), as an equal
+number of conflicts does not mean that they are the @emph{same}. When
+possible, you should rather use precedence directives to @emph{fix} the
+conflicts explicitly (@pxref{Non Operators,, Using Precedence For Non
+Operators}).
The definition of @code{if_stmt} above is solely to blame for the
conflict, but the conflict does not actually appear without additional
@example
@group
-%token IF THEN ELSE variable
%%
@end group
@group
@group
if_stmt:
- IF expr THEN stmt
-| IF expr THEN stmt ELSE stmt
+ "if" expr "then" stmt
+| "if" expr "then" stmt "else" stmt
;
@end group
expr:
- variable
+ "identifier"
;
@end example
* Precedence Only:: How to specify precedence only.
* Precedence Examples:: How these features are used in the previous example.
* How Precedence:: How they work.
+* Non Operators:: Using precedence for general conflicts.
@end menu
@node Why Precedence
declared with @code{'-'}:
@example
-%left '<' '>' '=' NE LE GE
+%left '<' '>' '=' "!=" "<=" ">="
%left '+' '-'
%left '*' '/'
@end example
-@noindent
-(Here @code{NE} and so on stand for the operators for ``not equal''
-and so on. We assume that these tokens are more than one character long
-and therefore are represented by names, not character literals.)
-
@node How Precedence
@subsection How Precedence Works
Not all rules and not all tokens have precedence. If either the rule or
the lookahead token has no precedence, then the default is to shift.
+@node Non Operators
+@subsection Using Precedence For Non Operators
+
+Using properly precedence and associativity directives can help fixing
+shift/reduce conflicts that do not involve arithmetics-like operators. For
+instance, the ``dangling @code{else}'' problem (@pxref{Shift/Reduce, ,
+Shift/Reduce Conflicts}) can be solved elegantly in two different ways.
+
+In the present case, the conflict is between the token @code{"else"} willing
+to be shifted, and the rule @samp{if_stmt: "if" expr "then" stmt}, asking
+for reduction. By default, the precedence of a rule is that of its last
+token, here @code{"then"}, so the conflict will be solved appropriately
+by giving @code{"else"} a precedence higher than that of @code{"then"}, for
+instance as follows:
+
+@example
+@group
+%nonassoc "then"
+%nonassoc "else"
+@end group
+@end example
+
+Alternatively, you may give both tokens the same precedence, in which case
+associativity is used to solve the conflict. To preserve the shift action,
+use right associativity:
+
+@example
+%right "then" "else"
+@end example
+
+Neither solution is perfect however. Since Bison does not provide, so far,
+support for ``scoped'' precedence, both force you to declare the precedence
+of these keywords with respect to the other operators your grammar.
+Therefore, instead of being warned about new conflicts you would be unaware
+of (e.g., a shift/reduce conflict due to @samp{if test then 1 else 2 + 3}
+being ambiguous: @samp{if test then 1 else (2 + 3)} or @samp{(if test then 1
+else 2) + 3}?), the conflict will be already ``fixed''.
+
@node Contextual Precedence
@section Context-Dependent Precedence
@cindex context-dependent precedence
proper way to define @code{sequence}:
@example
+@group
sequence:
/* empty */ @{ printf ("empty sequence\n"); @}
| sequence word @{ printf ("added word %s\n", $2); @}
;
+@end group
@end example
Here is another common error that yields a reduce/reduce conflict:
@example
sequence:
+@group
/* empty */
| sequence words
| sequence redirects
;
+@end group
+@group
words:
/* empty */
| words word
;
+@end group
+@group
redirects:
/* empty */
| redirects redirect
;
+@end group
@end example
@noindent
@end group
@end example
+Yet this proposal introduces another kind of ambiguity! The input
+@samp{word word} can be parsed as a single @code{words} composed of two
+@samp{word}s, or as two one-@code{word} @code{words} (and likewise for
+@code{redirect}/@code{redirects}). However this ambiguity is now a
+shift/reduce conflict, and therefore it can now be addressed with precedence
+directives.
+
+To simplify the matter, we will proceed with @code{word} and @code{redirect}
+being tokens: @code{"word"} and @code{"redirect"}.
+
+To prefer the longest @code{words}, the conflict between the token
+@code{"word"} and the rule @samp{sequence: sequence words} must be resolved
+as a shift. To this end, we use the same techniques as exposed above, see
+@ref{Non Operators,, Using Precedence For Non Operators}. One solution
+relies on precedences: use @code{%prec} to give a lower precedence to the
+rule:
+
+@example
+%nonassoc "word"
+%nonassoc "sequence"
+%%
+@group
+sequence:
+ /* empty */
+| sequence word %prec "sequence"
+| sequence redirect %prec "sequence"
+;
+@end group
+
+@group
+words:
+ word
+| words "word"
+;
+@end group
+@end example
+
+Another solution relies on associativity: provide both the token and the
+rule with the same precedence, but make them right-associative:
+
+@example
+%right "word" "redirect"
+%%
+@group
+sequence:
+ /* empty */
+| sequence word %prec "word"
+| sequence redirect %prec "redirect"
+;
+@end group
+@end example
+
@node Mysterious Conflicts
@section Mysterious Conflicts
@cindex Mysterious Conflicts
@example
@group
-%token ID
-
%%
def: param_spec return_spec ',';
param_spec:
;
@end group
@group
-type: ID;
+type: "id";
@end group
@group
-name: ID;
+name: "id";
name_list:
name
| name ',' name_list
@end group
@end example
-It would seem that this grammar can be parsed with only a single token
-of lookahead: when a @code{param_spec} is being read, an @code{ID} is
-a @code{name} if a comma or colon follows, or a @code{type} if another
-@code{ID} follows. In other words, this grammar is LR(1).
+It would seem that this grammar can be parsed with only a single token of
+lookahead: when a @code{param_spec} is being read, an @code{"id"} is a
+@code{name} if a comma or colon follows, or a @code{type} if another
+@code{"id"} follows. In other words, this grammar is LR(1).
@cindex LR
@cindex LALR
However, for historical reasons, Bison cannot by default handle all
LR(1) grammars.
-In this grammar, two contexts, that after an @code{ID} at the beginning
+In this grammar, two contexts, that after an @code{"id"} at the beginning
of a @code{param_spec} and likewise at the beginning of a
@code{return_spec}, are similar enough that Bison assumes they are the
same.
@example
@group
-%token BOGUS
-@dots{}
-%%
@dots{}
return_spec:
type
| name ':' type
-| ID BOGUS /* This rule is never used. */
+| "id" "bogus" /* This rule is never used. */
;
@end group
@end example
This corrects the problem because it introduces the possibility of an
-additional active rule in the context after the @code{ID} at the beginning of
+additional active rule in the context after the @code{"id"} at the beginning of
@code{return_spec}. This rule is not active in the corresponding context
in a @code{param_spec}, so the two contexts receive distinct parser states.
-As long as the token @code{BOGUS} is never generated by @code{yylex},
+As long as the token @code{"bogus"} is never generated by @code{yylex},
the added rule cannot alter the way actual input is parsed.
In this particular example, there is another way to solve the problem:
-rewrite the rule for @code{return_spec} to use @code{ID} directly
+rewrite the rule for @code{return_spec} to use @code{"id"} directly
instead of via @code{name}. This also causes the two confusing
contexts to have different sets of active rules, because the one for
@code{return_spec} activates the altered rule for @code{return_spec}
;
return_spec:
type
-| ID ':' type
+| "id" ':' type
;
@end example
@c LocalWords: subdirectory Solaris nonassociativity perror schemas Malloy ints
@c LocalWords: Scannerless ispell american ChangeLog smallexample CSTYPE CLTYPE
@c LocalWords: clval CDEBUG cdebug deftypeopx yyterminate LocationType
-@c LocalWords: errorVerbose
+@c LocalWords: parsers parser's
+@c LocalWords: associativity subclasses precedences unresolvable runnable
+@c LocalWords: allocators subunit initializations unreferenced untyped
+@c LocalWords: errorVerbose subtype subtypes
@c Local Variables:
@c ispell-dictionary: "american"
%x SC_BRACKETED_ID SC_RETURN_BRACKETED_ID
letter [.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_]
+notletter [^.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_]{-}[%\{]
id {letter}({letter}|[-0-9])*
-directive %{id}
int [0-9]+
/* POSIX says that a tag must be both an id and a C union member, but
complain (loc, Wother, _("stray ',' treated as white space"));
}
[ \f\n\t\v] |
- "//".* ;
+ "//".* continue;
"/*" {
token_start = loc->start;
context_state = YY_START;
"%pure"[-_]"parser" DEPRECATED("%pure-parser");
"%token"[-_]"table" DEPRECATED("%token-table");
- {directive} {
+ "%"{id}|"%"{notletter}([[:graph:]])+ {
complain (loc, complaint, _("invalid directive: %s"), quote (yytext));
}
BEGIN SC_BRACKETED_ID;
}
- . {
- complain (loc, complaint, _("invalid character: %s"),
+ [^\[%A-Za-z0-9_<>{}\"\'*;|=/, \f\n\t\v]+|. {
+ complain (loc, complaint, "%s: %s",
+ ngettext ("invalid character", "invalid characters", yyleng),
quote_mem (yytext, yyleng));
}
else
complain (loc, complaint, _("an identifier expected"));
}
- . {
- complain (loc, complaint, _("invalid character in bracketed name: %s"),
+
+ [^\].A-Za-z0-9_/ \f\n\t\v]+|. {
+ complain (loc, complaint, "%s: %s",
+ ngettext ("invalid character in bracketed name",
+ "invalid characters in bracketed name", yyleng),
quote_mem (yytext, yyleng));
}
+
<<EOF>> {
BEGIN bracketed_id_context_state;
unexpected_eof (bracketed_id_start, "]");
<SC_YACC_COMMENT>
{
"*/" BEGIN context_state;
- .|\n ;
+ .|\n continue;
<<EOF>> unexpected_eof (token_start, "*/"); BEGIN context_state;
}
%%
input:
line
-| input line { }
+| input line {}
;
line:
'\n'
-| exp '\n' { }
+| exp '\n' {}
;
exp:
$$ = $l;
}
| exp[x] '+' { $<ival>$ = $x; } [l] exp[r] { $$ = $<ival>l + $r; }
-| exp[l] '-' exp[r] { $$ = $l - $r; }
-| exp[l] '*' exp[r] { $$ = $l * $r; }
+| exp[l] '-' exp[r] { $$ = $l - $r; }
+| exp[l] '*' exp[r] { $$ = $l * $r; }
| exp[l] '/' exp[r] { $$ = $l / $r; }
| '-' exp %prec NEG { $$ = -$2; }
-| exp[l] '^' exp[r] { $$ = power ($l, $r); }
-| '(' exp[e] ')' { $$ = $e; }
+| exp[l] '^' exp[r] { $$ = power ($l, $r); }
+| '(' exp[e] ')' { $$ = $e; }
| '(' error ')' { $$ = 1111; yyerrok; }
| '!' { $$ = 0; YYERROR; }
| '-' error { $$ = 0; YYERROR; }
%%
input:
line
-| input line { }
+| input line {}
;
line:
'\n'
-| exp '\n' { }
+| exp '\n' {}
;
exp:
| exp[x] '*' { $<ival>$ = $x; } [l] exp[r] { $$ = $l * $r; }
| exp[l] '/' exp[r] { $$ = $l / $r; }
| '-' exp %prec NEG { $$ = -$2; }
-| exp[l] '^' exp[r] { $$ = power ($l, $r12); }
+| exp[l] '^' exp[r] { $$ = power ($l, $r12); }
| '(' exp ')' { $$ = $expo; }
| '(' error ')' { $$ = 1111; yyerrok; }
| '!' { $$ = 0; YYERROR; }
test.y:51.7: possibly meant: $x, hiding $exp at $1
test.y:51.41: possibly meant: $r, hiding $exp at $4
test.y:52.51-52: error: $l of 'exp' has no declared type
-test.y:55.46-49: error: invalid reference: '$r12'
-test.y:55.3-53: symbol not found in production: r12
+test.y:55.40-43: error: invalid reference: '$r12'
+test.y:55.3-47: symbol not found in production: r12
test.y:56.29-33: error: invalid reference: '$expo'
test.y:56.3-46: symbol not found in production: expo
]])
AT_DATA_GRAMMAR([test.y],
[[
%%
-start: foo[ /* aaa */ *&-.+\000\001\002\377 ] bar
+start: foo[ % /* aaa */ *&-.+\000\001\002\377 ] bar
{ s = $foo; }
]])
AT_CHECK([[$PERL -pi -e 's/\\(\d{3})/chr(oct($1))/ge' test.y || exit 77]])
AT_BISON_CHECK([-o test.c test.y], 1, [],
-[[test.y:11.23: error: invalid character in bracketed name: '*'
-test.y:11.24: error: invalid character in bracketed name: '&'
-test.y:11.25: error: invalid character in bracketed name: '-'
-test.y:11.27: error: invalid character in bracketed name: '+'
-test.y:11.28: error: invalid character in bracketed name: '\0'
-test.y:11.28: error: invalid character in bracketed name: '\001'
-test.y:11.28: error: invalid character in bracketed name: '\002'
-test.y:11.28: error: invalid character in bracketed name: '\377'
+[[test.y:11.13: error: invalid character in bracketed name: '%'
+test.y:11.25-27: error: invalid characters in bracketed name: '*&-'
+test.y:11.29-30: error: invalid characters in bracketed name: '+\0\001\002\377'
]])
AT_CLEANUP
projects / bison.git / commitdiff
