doc: token.prefix

[bison.git] / HACKING

-*- outline -*-

This file attempts to describe the rules to use when hacking Bison.
Don't put this file into the distribution.

Everything related to the development of Bison is on Savannah:

        http://savannah.gnu.org/projects/bison/


* Administrivia

** If you incorporate a change from somebody on the net:
First, if it is a large change, you must make sure they have signed
the appropriate paperwork.  Second, be sure to add their name and
email address to THANKS.

** If a change fixes a test, mention the test in the ChangeLog entry.

** Bug reports
If somebody reports a new bug, mention his name in the ChangeLog entry
and in the test case you write.  Put him into THANKS.

The correct response to most actual bugs is to write a new test case
which demonstrates the bug.  Then fix the bug, re-run the test suite,
and check everything in.

** You may find it useful to install the git-merge-changelog merge driver:

  http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=blob;f=lib/git-merge-changelog.c

When following the generic installation instructions there, keep in mind that
your clone of Bison's git repository already contains appropriate
.gitattributes files, and running Bison's bootstrap script will make the
necessary changes to .git/config.


* Hacking

** Visible changes
Which include serious bug fixes, must be mentioned in NEWS.

** Translations
Only user visible strings are to be translated: error messages, bits
of the .output file etc.  This excludes impossible error messages
(comparable to assert/abort), and all the --trace output which is
meant for the maintainers only.


* Working from the repository

These notes intend to help people working on the checked-out sources.
These requirements do not apply when building from a distribution tarball.

** Requirements

We've opted to keep only the highest-level sources in the repository.
This eases our maintenance burden, (fewer merges etc.), but imposes more
requirements on anyone wishing to build from the just-checked-out sources.
For example, you have to use the latest stable versions of the maintainer
tools we depend upon, including:

- Automake <http://www.gnu.org/software/automake/>
- Autoconf <http://www.gnu.org/software/autoconf/>
- Flex <http://www.gnu.org/software/flex/>
- Gettext <http://www.gnu.org/software/gettext/>
- Gzip <http://www.gnu.org/software/gzip/>
- Perl <http://www.cpan.org/>
- Rsync <http://samba.anu.edu.au/rsync/>
- Tar <http://www.gnu.org/software/tar/>

Valgrind <http://valgrind.org/> is also highly recommended, if
Valgrind supports your architecture.

Bison is written using Bison grammars, so there are bootstrapping
issues.  The bootstrap script attempts to discover when the C code
generated from the grammars is out of date, and to bootstrap with an
out-of-date version of the C code, but the process is not foolproof.
Also, you may run into similar problems yourself if you modify Bison.

Only building the initial full source tree will be a bit painful.
Later, after synchronizing from the repository a plain `make' should
be sufficient.

** First checkout

Obviously, if you are reading these notes, you did manage to check out
this package from the repository.  For the record, you will find all the
relevant information on:

        http://savannah.gnu.org/git/?group=bison

Bison uses Git submodules: subscriptions to other Git repositories.
In particular it uses gnulib, the GNU portability library.  To ask Git
to perform the first checkout of the submodules, run

       $ git submodule update --init

Git submodule support is weak before versions 1.6 and later, you
should probably upgrade Git if your version is older.

The next step is to get other files needed to build, which are
extracted from other source packages:

        $ ./bootstrap

And there you are!  Just

        $ ./configure
        $ make
        $ make check

At this point, there should be no difference between your local copy,
and the master copy:

        $ git diff

should output no difference.

Enjoy!

** Updating

The use of submodules make things somewhat different because git does
not support recursive operations: submodules must be taken care of
explicitly by the user.

*** Updating Bison

If you pull a newer version of a branch, say via `git pull', you might
import requests for updated submodules.  A simple `git diff' will
reveal if the current version of the submodule (i.e., the actual
contents of the gnulib directory) and the current request from the
subscriber (i.e., the reference of the version of gnulib that the
Bison reporitory requests) differ.  To upgrade the submodules (i.e.,
to check out the version that is actually requested by the subscriber,
run `git submodule update'.

        $ git pull
        $ git submodule update

*** Updating a submodule
To update a submodule, say gnulib, do as follows:

Get the most recent version of the master branch from git.

        $ cd gnulib
        $ git fetch
        $ git checkout -b master --track origin/master

Make sure Bison can live with that version of gnulib.

        $ cd ..
        $ ./bootstrap
        $ make distcheck

Register your changes.

        $ git checkin ...


* Test suite

** make check
Use liberally.

** Release checks
Try to run the test suite with more severe conditions before a
release:

- Configure the package with --enable-gcc-warnings, so that one checks
  that 1. Bison compiles cleanly, 2. the parsers it produces compile
  cleanly too.

- run `make maintainer-check' which:
  - runs `valgrind -q bison' to run Bison under Valgrind.
  - runs the parsers under Valgrind.
  - runs the test suite with G++ as C compiler...

- run `make maintainer-push-check', which runs `make maintainer-check'
  while activating the push implementation and its pull interface wrappers
  in many test cases that were originally written to exercise only the
  pull implementation.  This makes certain the push implementation can
  perform every task the pull implementation can.

- run `make maintainer-xml-check', which runs `make maintainer-check'
  while checking Bison's XML automaton report for every working grammar
  passed to Bison in the test suite.  The check just diffs the output of
  Bison's included XSLT style sheets with the output of --report=all and
  --graph.

- Change tests/atlocal/CFLAGS to add your preferred options.  For
  instance, `-traditional' to check that the parsers are K&R.  Note
  that it does not make sense for glr.c, which should be ANSI,
  but currently is actually GNU C, nor for lalr1.cc.


* Release Procedure

** Try to get the *.pot files to the Translation Project at least one week
before a stable release, to give them time to translate them.

** Tests
See above.

** Update the foreign files
Running `./bootstrap' in the top level should update them all for you.
This covers PO files too.  Sometimes a PO file contains problems that
causes it to be rejected by recent Gettext releases; please report
these to the Translation Project.

** Update README
Make sure the information in this file is current.  Most notably, make sure it
recommends a version of GNU M4 that is compatible with the latest Bison
sources.

** Update NEWS
The version number, *and* the date of the release (including for
betas).

** Update ChangeLog
Should have an entry similar to `Version 1.49b.'.

** Update configure.ac
Be sure PACKAGE_COPYRIGHT_YEAR is up-to-date.

** Tag the release
Before Bison will build with the right version number, you must tag the release
in git.  Do this after all other changes.  The command is similar to:

  git tag -a v2.3b

The log message can be simply:

  Bison 2.3b

** Push
Once `make distcheck' passes, push your changes and the tag.
`git push' without arguments will not push the tag.

** make alpha
FIXME: `make alpha' is not maintained and is broken.  These
instructions need to be replaced or removed.

Running `make alpha' is absolutely perfect for beta releases: it makes
the tarballs, the xdeltas, and prepares (in /tmp/) a proto
announcement.  It is so neat, that that's what I use anyway for
genuine releases, but adjusting things by hand (e.g., the urls in the
announcement file, the ChangeLog which is not needed etc.).

If it fails, you're on your own...

It requires GNU Make.

** Upload
The generic GNU upload procedure is at:

  http://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads

Follow the instructions there to register your information so you're permitted
to upload.  Make sure your public key has been uploaded at least to
keys.gnupg.net.  You can upload it with:

  gpg --keyserver keys.gnupg.net --send-keys F125BDF3

where F125BDF3 should be replaced with your key ID.

Here's a brief reminder of how to roll the tarballs and upload them:

*** make distcheck
*** gpg -b bison-2.3b.tar.gz
*** In a file named `bison-2.3b.tar.gz.directive', type:

      version: 1.1
      directory: bison
      filename: bison-2.3b.tar.gz

*** gpg --clearsign bison-2.3b.tar.gz.directive
*** ftp ftp-upload.gnu.org # Log in as anonymous.
*** cd /incoming/alpha # cd /incoming/ftp for full release.
*** put bison-2.3b.tar.gz # This can take a while.
*** put bison-2.3b.tar.gz.sig
*** put bison-2.3b.tar.gz.directive.asc
*** Repeat all these steps for bison-2.3b.tar.bz2.

** Update Bison manual on www.gnu.org.

*** You need a non-anonymous checkout of the web pages directory.

  $ cvs -d YOUR_USERID@cvs.savannah.gnu.org:/web/bison checkout bison

*** Get familiar with the instructions for web page maintainers.
http://www.gnu.org/server/standards/readme_index.html
http://www.gnu.org/server/standards/README.software.html
especially the note about symlinks.

*** Build the web pages.
Assuming BISON_CHECKOUT refers to a checkout of the Bison dir, and
BISON_WWW_CHECKOUT refers to the web directory created above, do:

  $ cd $BISON_CHECKOUT/doc
  $ make stamp-vti
  $ ../build-aux/gendocs.sh -o "$BISON_WWW_CHECKOUT/manual" \
    bison "Bison - GNU parser generator"
  $ cd $BISON_WWW_CHECKOUT

Verify that the result looks sane.

*** Commit the modified and the new files.

*** Remove old files.
Find the files which have not been overwritten (because they belonged to
sections that have been removed or renamed):

   $ cd manual/html_node
   $ ls -lt

Remove these files and commit their removal to CVS.  For each of these
files, add a line to the file .symlinks.  This will ensure that
hyperlinks to the removed files will redirect to the entire manual; this
is better than a 404 error.

There is a problem with 'index.html' being written twice (once for POSIX
function 'index', once for the table of contents); you can ignore this
issue.

** Announce
To generate a template announcement file:

  make RELEASE_TYPE=alpha gpg_key_ID=F125BDF3 announcement

where alpha can be replaced by beta or major and F125BDF3 should be replaced
with your key ID.  For an example of how to fill out the template, search the
mailing list archives for the most recent release announcement.

Complete/fix the announcement file, and send it at least to
info-gnu@gnu.org (if a real release, or a ``serious beta''),
bug-bison@gnu.org, help-bison@gnu.org, bison-patches@gnu.org,
and coordinator@translationproject.org.

Send the same announcement on the comp.compilers newsgroup by sending
email to compilers@iecc.com.  Do not make any Cc as the moderator will
throw away anything cross-posted or Cc'ed.  It really needs to be a
separate message.

** Bump the version number
In configure.ac.  Run `make'.  So that developers don't accidentally add new
items to the old NEWS entry, create a new empty NEWS entry something like:

  Changes in version ?.? (????-??-??):

Push these changes.


-----

Copyright (C) 2002, 2003, 2004, 2005, 2007, 2008, 2009
Free Software Foundation, Inc.

This file is part of GNU Bison.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.