More.

[bison.git] / TODO

-*- outline -*-

* Coding system independence
Paul notes:

	Currently Bison assumes 8-bit bytes (i.e. that UCHAR_MAX is
	255).  It also assumes that the 8-bit character encoding is
	the same for the invocation of 'bison' as it is for the
	invocation of 'cc', but this is not necessarily true when
	people run bison on an ASCII host and then use cc on an EBCDIC
	host.  I don't think these topics are worth our time
	addressing (unless we find a gung-ho volunteer for EBCDIC or
	PDP-10 ports :-) but they should probably be documented
	somewhere.

* Using enums instead of int for tokens.
Paul suggests:

   #ifndef YYTOKENTYPE
   # if defined (__STDC__) || defined (__cplusplus)
      /* Put the tokens into the symbol table, so that GDB and other debuggers
         know about them.  */
      enum yytokentype {
        FOO = 256,
        BAR,
        ...
      };
      /* POSIX requires `int' for tokens in interfaces.  */
   #  define YYTOKENTYPE int
   # endif
   #endif
   #define FOO 256
   #define BAR 257
   ...

> I'm in favor of
>
> %token FOO 256
> %token BAR 257
>
> and Bison moves error into 258.

Yes, I think that's a valid extension too, if the user doesn't define
the token number for error.

* Unit rules
Maybe we could expand unit rules, i.e., transform

	exp: arith | bool;
	arith: exp '+' exp;
	bool: exp '&' exp;

into

	exp: exp '+' exp | exp '&' exp;

when there are no actions.  This can significantly speed up some
grammars.

* Stupid error messages
An example shows it easily:

src/bison/tests % ./testsuite -k calc,location,error-verbose -l
GNU Bison 1.49a test suite test groups:

 NUM: FILENAME:LINE      TEST-GROUP-NAME
      KEYWORDS

  51: calc.at:440        Calculator --locations --yyerror-verbose
  52: calc.at:442        Calculator --defines --locations --name-prefix=calc --verbose --yacc --yyerror-verbose
  54: calc.at:445        Calculator --debug --defines --locations --name-prefix=calc --verbose --yacc --yyerror-verbose
src/bison/tests % ./testsuite 51 -d
## --------------------------- ##
## GNU Bison 1.49a test suite. ##
## --------------------------- ##
 51: calc.at:440       ok
## ---------------------------- ##
## All 1 tests were successful. ##
## ---------------------------- ##
src/bison/tests % cd ./testsuite.dir/51
tests/testsuite.dir/51 % echo "()" | ./calc
1.2-1.3: parse error, unexpected ')', expecting error or "number" or '-' or '('

* read_pipe.c
This is not portable to DOS for instance.  Implement a more portable
scheme.  Sources of inspiration include GNU diff, and Free Recode.

* Memory leaks in the generator
A round of memory leak clean ups would be most welcome.  Dmalloc,
Checker GCC, Electric Fence, or Valgrind: you chose your tool.

* Memory leaks in the parser
The same applies to the generated parsers.  In particular, this is
critical for user data: when aborting a parsing, when handling the
error token etc., we often throw away yylval without giving a chance
of cleaning it up to the user.

* NEWS
Sort from 1.31 NEWS.

* Prologue
The %union is declared after the user C declarations. It can be
a problem if YYSTYPE is declared after the user part.	[]

Actually, the real problem seems that the %union ought to be output
where it was defined.  For instance, in gettext/intl/plural.y, we
have:

	%{
	...
	#include "gettextP.h"
	...
	%}

	%union {
	  unsigned long int num;
	  enum operator op;
	  struct expression *exp;
	}

	%{
	...
	static int yylex PARAMS ((YYSTYPE *lval, const char **pexp));
	...
	%}

Where the first part defines struct expression, the second uses it to
define YYSTYPE, and the last uses YYSTYPE.  Only this order is valid.

* --graph
Show reductions.	[]

* Broken options ?
** %no-lines		[ok]
** %no-parser		[]
** %pure-parser		[]
** %semantic-parser	[]
** %token-table		[]
** Options which could use parse_dquoted_param ().
Maybe transfered in lex.c.
*** %skeleton		[ok]
*** %output		[]
*** %file-prefix	[]
*** %name-prefix	[]

** Skeleton strategy.	[]
Must we keep %no-parser?
	     %token-table?
*** New skeletons.	[]

* src/print_graph.c
Find the best graph parameters.	[]

* doc/bison.texinfo
** Update
informations about ERROR_VERBOSE.	[]
** Add explainations about
skeleton muscles.	[]
%skeleton.		[]

* testsuite
** tests/pure-parser.at	[]
New tests.

* Debugging parsers

From Greg McGary:

akim demaille <akim.demaille@epita.fr> writes:

> With great pleasure!  Nonetheless, things which are debatable
> (or not, but just `big') should be discuss in `public': something
> like help- or bug-bison@gnu.org is just fine.  Jesse and I are there,
> but there is also Jim and some other people.

I have no idea whether it qualifies as big or controversial, so I'll
just summarize for you.  I proposed this change years ago and was
surprised that it was met with utter indifference!

This debug feature is for the programs/grammars one develops with
bison, not for debugging bison itself.  I find that the YYDEBUG
output comes in a very inconvenient format for my purposes.
When debugging gcc, for instance, what I want is to see a trace of
the sequence of reductions and the line#s for the semantic actions
so I can follow what's happening.  Single-step in gdb doesn't cut it
because to move from one semantic action to the next takes you through
lots of internal machinery of the parser, which is uninteresting.

The change I made was to the format of the debug output, so that it
comes out in the format of C error messages, digestible by emacs
compile mode, like so:

grammar.y:1234: foo: bar(0x123456) baz(0x345678)

where "foo: bar baz" is the reduction rule, whose semantic action
appears on line 1234 of the bison grammar file grammar.y.  The hex
numbers on the rhs tokens are the parse-stack values associated with
those tokens.  Of course, yytype might be something totally
incompatible with that representation, but for the most part, yytype
values are single words (scalars or pointers).  In the case of gcc,
they're most often pointers to tree nodes.  Come to think of it, the
right thing to do is to make the printing of stack values be
user-definable.  It would also be useful to include the filename &
line# of the file being parsed, but the main filename & line# should
continue to be that of grammar.y

Anyway, this feature has saved my life on numerous occasions.  The way
I customarily use it is to first run bison with the traces on, isolate
the sequence of reductions that interests me, put those traces in a
buffer and force it into compile-mode, then visit each of those lines
in the grammar and set breakpoints with C-x SPACE.  Then, I can run
again under the control of gdb and stop at each semantic action.
With the hex addresses of tree nodes, I can inspect the values
associated with any rhs token.

You like?

* input synclines
Some users create their foo.y files, and equip them with #line.  Bison
should recognize these, and preserve them.

* BTYacc
See if we can integrate backtracking in Bison.  Contact the BTYacc
maintainers.

* Automaton report
Display more clearly the lookaheads for each item.

* RR conflicts
See if we can use precedence between rules to solve RR conflicts.  See
what POSIX says.

* Precedence
It is unfortunate that there is a total order for precedence.  It
makes it impossible to have modular precedence information.  We should
move to partial orders.

* Parsing grammars
Rewrite the reader in Bison.