tests: minor refactoring

[bison.git] / README-hacking

diff --git a/README-hacking b/README-hacking
index 6a24062490beab5274eab83d128849494a885470..28870cb268de91ad3704123dff12769b9fd135ad 100644 (file)
--- a/README-hacking
+++ b/README-hacking
@@ -1,5 +1,3 @@
--*- outline -*-
-

 This file attempts to describe the rules to use when hacking Bison.
 Don't put this file into the distribution.
 
 This file attempts to describe the rules to use when hacking Bison.
 Don't put this file into the distribution.
 


@@ -15,25 +13,16 @@ First, if it is a large change, you must make sure they have signed
 the appropriate paperwork.  Second, be sure to add their name and
 email address to THANKS.
 
 the appropriate paperwork.  Second, be sure to add their name and
 email address to THANKS.
 

-** If a change fixes a test, mention the test in the ChangeLog entry.
+** If a change fixes a test, mention the test in the commit message.

 
 ** Bug reports
 
 ** Bug reports

-If somebody reports a new bug, mention his name in the ChangeLog entry
+If somebody reports a new bug, mention his name in the commit message

 and in the test case you write.  Put him into THANKS.
 
 The correct response to most actual bugs is to write a new test case
 which demonstrates the bug.  Then fix the bug, re-run the test suite,
 and check everything in.
 
 and in the test case you write.  Put him into THANKS.
 
 The correct response to most actual bugs is to write a new test case
 which demonstrates the bug.  Then fix the bug, re-run the test suite,
 and check everything in.
 

-** You may find it useful to install the git-merge-changelog merge driver:
-
-  http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=blob;f=lib/git-merge-changelog.c
-
-When following the generic installation instructions there, keep in mind that
-your clone of Bison's git repository already contains appropriate
-.gitattributes files, and running Bison's bootstrap script will make the
-necessary changes to .git/config.
-

 
 * Hacking
 
 
 * Hacking
 


@@ -46,6 +35,13 @@ of the .output file etc.  This excludes impossible error messages
 (comparable to assert/abort), and all the --trace output which is
 meant for the maintainers only.
 
 (comparable to assert/abort), and all the --trace output which is
 meant for the maintainers only.
 

+** Horizontal tabs
+Do not add horizontal tab characters to any file in Bison's repository
+except where required.  For example, do not use tabs to format C code.
+However, make files, ChangeLog, and some regular expressions require
+tabs.  Also, test cases might need to contain tabs to check that Bison
+properly processes tabs in its input.
+

 
 * Working from the repository
 
 
 * Working from the repository
 


@@ -54,33 +50,35 @@ These requirements do not apply when building from a distribution tarball.
 
 ** Requirements
 
 
 ** Requirements
 

-We've opted to keep only the highest-level sources in the repository.
-This eases our maintenance burden, (fewer merges etc.), but imposes more
+We've opted to keep only the highest-level sources in the repository.  This
+eases our maintenance burden, (fewer merges etc.), but imposes more

 requirements on anyone wishing to build from the just-checked-out sources.
 For example, you have to use the latest stable versions of the maintainer
 tools we depend upon, including:
 
 requirements on anyone wishing to build from the just-checked-out sources.
 For example, you have to use the latest stable versions of the maintainer
 tools we depend upon, including:
 

-- Automake <http://www.gnu.org/software/automake/>

 - Autoconf <http://www.gnu.org/software/autoconf/>
 - Autoconf <http://www.gnu.org/software/autoconf/>

+- Automake <http://www.gnu.org/software/automake/>

 - Flex <http://www.gnu.org/software/flex/>
 - Gettext <http://www.gnu.org/software/gettext/>
 - Flex <http://www.gnu.org/software/flex/>
 - Gettext <http://www.gnu.org/software/gettext/>

+- Graphviz <http://www.graphviz.org>

 - Gzip <http://www.gnu.org/software/gzip/>
 - Perl <http://www.cpan.org/>
 - Rsync <http://samba.anu.edu.au/rsync/>
 - Tar <http://www.gnu.org/software/tar/>
 
 - Gzip <http://www.gnu.org/software/gzip/>
 - Perl <http://www.cpan.org/>
 - Rsync <http://samba.anu.edu.au/rsync/>
 - Tar <http://www.gnu.org/software/tar/>
 

-Valgrind <http://valgrind.org/> is also highly recommended, if
-Valgrind supports your architecture.
+Valgrind <http://valgrind.org/> is also highly recommended, if it supports
+your architecture.

 
 

-Bison is written using Bison grammars, so there are bootstrapping
-issues.  The bootstrap script attempts to discover when the C code
-generated from the grammars is out of date, and to bootstrap with an
-out-of-date version of the C code, but the process is not foolproof.
-Also, you may run into similar problems yourself if you modify Bison.
+Bison is written using Bison grammars, so there are bootstrapping issues.
+The bootstrap script attempts to discover when the C code generated from the
+grammars is out of date, and to bootstrap with an out-of-date version of the
+C code, but the process is not foolproof.  Also, you may run into similar
+problems yourself if you modify Bison.

 
 

-Only building the initial full source tree will be a bit painful.
-Later, after synchronizing from the repository a plain `make' should
-be sufficient.
+Only building the initial full source tree will be a bit painful.  Later,
+after synchronizing from the repository a plain 'make' should be sufficient.
+Note, however, that when gnulib is updated, running './bootstrap' again
+might be needed.

 
 ** First checkout
 
 
 ** First checkout
 


@@ -127,14 +125,14 @@ explicitly by the user.
 
 *** Updating Bison
 
 
 *** Updating Bison
 

-If you pull a newer version of a branch, say via `git pull', you might
-import requests for updated submodules.  A simple `git diff' will
+If you pull a newer version of a branch, say via "git pull", you might
+import requests for updated submodules.  A simple "git diff" will

 reveal if the current version of the submodule (i.e., the actual
 contents of the gnulib directory) and the current request from the
 subscriber (i.e., the reference of the version of gnulib that the
 reveal if the current version of the submodule (i.e., the actual
 contents of the gnulib directory) and the current request from the
 subscriber (i.e., the reference of the version of gnulib that the

-Bison reporitory requests) differ.  To upgrade the submodules (i.e.,
+Bison repository requests) differ.  To upgrade the submodules (i.e.,

 to check out the version that is actually requested by the subscriber,
 to check out the version that is actually requested by the subscriber,

-run `git submodule update'.
+run "git submodule update".

 
         $ git pull
         $ git submodule update
 
         $ git pull
         $ git submodule update


@@ -163,13 +161,13 @@ formal release, see the ChangeLog in the latest gnulib snapshot at:
 
         http://erislabs.net/ianb/projects/gnulib/
 
 
         http://erislabs.net/ianb/projects/gnulib/
 

-The autoconf files we use are currently:
+The Autoconf files we use are currently:

 
         m4/m4.m4
         lib/m4sugar/m4sugar.m4
         lib/m4sugar/foreach.m4
 
 
         m4/m4.m4
         lib/m4sugar/m4sugar.m4
         lib/m4sugar/foreach.m4
 

-These files don't change very often in autoconf, so it should be
+These files don't change very often in Autoconf, so it should be

 relatively straight-forward to examine the differences in order to
 decide whether to update.
 
 relatively straight-forward to examine the differences in order to
 decide whether to update.
 


@@ -178,6 +176,58 @@ decide whether to update.
 ** make check
 Use liberally.
 
 ** make check
 Use liberally.
 

+** TESTSUITEFLAGS
+
+The default is for make check to run all tests sequentially. This can be
+very time consumming when checking repeatedly or on slower setups. This can
+be sped up in two ways:
+
+Using -j, in a make-like fashion, for example:
+  $ make check TESTSUITEFLAGS='-j8'
+
+Running only the tests of a certain category, as specified in the AT files
+with AT_KEYWORDS([[category]]). Categories include:
+  - c++, for c++ parsers
+  - deprec, for tests concerning deprecated constructs.
+  - glr, for glr parsers
+  - java, for java parsers
+  - report, for automaton dumps
+
+To run a specific set of tests, use -k (for "keyword"). For example:
+  $ make check TESTSUITEFLAGS='-k c++'
+
+Both can be combined.
+
+** Typical errors
+If the test suite shows failures such as the following one
+
+  .../bison/lib/getopt.h:196:8: error: redefinition of 'struct option'
+  /usr/include/getopt.h:54:8: error: previous definition of 'struct option'
+
+it probably means that some file was compiled without
+AT_DATA_SOURCE_PROLOGUE.  This error is due to the fact that our -I options
+pick up gnulib's replacement headers, such as getopt.h, and this will go
+wrong if config.h was not included first.
+
+See tests/local.at for details.
+
+** make maintainer-check-valgrind
+This target uses valgrind both to check bison, and the generated parsers.
+
+This is not mature on Mac OS X.  First, Valgrind does support the way bison
+calls m4, so Valgrind cannot be used to check bison on Mac OS X.
+
+Second, there are many errors that come from the platform itself, not from
+bison.  build-aux/darwin11.4.0.valgrind addresses some of them.
+
+Third, valgrind issues warnings such as:
+
+  --99312:0:syswrap- WARNING: Ignoring sigreturn( ..., UC_RESET_ALT_STACK );
+
+which cause the test to fail uselessly.  It is hard to ignore these errors
+with a major overhaul of the way instrumentation is performed in the test
+suite.  So currently, do not try to run valgrind on Mac OS X.
+

 ** Release checks
 Try to run the test suite with more severe conditions before a
 release:
 ** Release checks
 Try to run the test suite with more severe conditions before a
 release:


@@ -186,36 +236,46 @@ release:
   that 1. Bison compiles cleanly, 2. the parsers it produces compile
   cleanly too.
 
   that 1. Bison compiles cleanly, 2. the parsers it produces compile
   cleanly too.
 

-- Build with -DGNULIB_POSIXCHECK.  It suggests gnulib modules that can
-  fix portability issues.
+- Maybe build with -DGNULIB_POSIXCHECK, which suggests gnulib modules
+  that can fix portability issues.  See if you really want to pay
+  attention to its warnings; there's no need to obey blindly to it
+  (<http://lists.gnu.org/archive/html/bison-patches/2012-05/msg00057.html>).

 
 

-- run `make maintainer-check' which:
-  - runs `valgrind -q bison' to run Bison under Valgrind.
+- Check with "make syntax-check" if there are issues diagnosed by
+  gnulib.
+
+- run "make maintainer-check" which:
+  - runs "valgrind -q bison" to run Bison under Valgrind.

   - runs the parsers under Valgrind.
   - runs the test suite with G++ as C compiler...
 
   - runs the parsers under Valgrind.
   - runs the test suite with G++ as C compiler...
 

-- run `make maintainer-push-check', which runs `make maintainer-check'
+- run "make maintainer-push-check", which runs "make maintainer-check"

   while activating the push implementation and its pull interface wrappers
   in many test cases that were originally written to exercise only the
   pull implementation.  This makes certain the push implementation can
   perform every task the pull implementation can.
 
   while activating the push implementation and its pull interface wrappers
   in many test cases that were originally written to exercise only the
   pull implementation.  This makes certain the push implementation can
   perform every task the pull implementation can.
 

-- run `make maintainer-xml-check', which runs `make maintainer-check'
+- run "make maintainer-xml-check", which runs "make maintainer-check"

   while checking Bison's XML automaton report for every working grammar
   passed to Bison in the test suite.  The check just diffs the output of
   Bison's included XSLT style sheets with the output of --report=all and
   --graph.
 
   while checking Bison's XML automaton report for every working grammar
   passed to Bison in the test suite.  The check just diffs the output of
   Bison's included XSLT style sheets with the output of --report=all and
   --graph.
 

+- running "make maintainer-release-check" takes care of running
+  maintainer-check, maintainer-push-check and maintainer-xml-check.
+

 - Change tests/atlocal/CFLAGS to add your preferred options.  For
 - Change tests/atlocal/CFLAGS to add your preferred options.  For

-  instance, `-traditional' to check that the parsers are K&R.  Note
-  that it does not make sense for glr.c, which should be ANSI,
-  but currently is actually GNU C, nor for lalr1.cc.
+  instance, "-traditional" to check that the parsers are K&R.  Note
+  that it does not make sense for glr.c, which should be ANSI, but
+  currently is actually GNU C, nor for lalr1.cc.

 
 - Test with a very recent version of GCC for both C and C++.  Testing
   with older versions that are still in use is nice too.
 
 
 * Release Procedure
 
 - Test with a very recent version of GCC for both C and C++.  Testing
   with older versions that are still in use is nice too.
 
 
 * Release Procedure

+This section needs to be updated to take into account features from
+gnulib.  In particular, be sure to read README-release.

 
 ** Update the submodules.  See above.
 
 
 ** Update the submodules.  See above.
 


@@ -231,7 +291,7 @@ This helps: grep -l '\<_(' *
 See above.
 
 ** Update the foreign files
 See above.
 
 ** Update the foreign files

-Running `./bootstrap' in the top level should update them all for you.
+Running "./bootstrap" in the top level should update them all for you.

 This covers PO files too.  Sometimes a PO file contains problems that
 causes it to be rejected by recent Gettext releases; please report
 these to the Translation Project.
 This covers PO files too.  Sometimes a PO file contains problems that
 causes it to be rejected by recent Gettext releases; please report
 these to the Translation Project.


@@ -243,65 +303,56 @@ Bison sources.
 
 ** Check copyright years.
 We update years in copyright statements throughout Bison once at the
 
 ** Check copyright years.
 We update years in copyright statements throughout Bison once at the

-start of every year by running `make update-copyright'.  However, before
+start of every year by running "make update-copyright".  However, before

 a release, it's good to verify that it's actually been run.  Besides the
 copyright statement for each Bison file, check the copyright statements
 that the skeletons insert into generated parsers, and check all
 occurrences of PACKAGE_COPYRIGHT_YEAR in configure.ac.
 
 a release, it's good to verify that it's actually been run.  Besides the
 copyright statement for each Bison file, check the copyright statements
 that the skeletons insert into generated parsers, and check all
 occurrences of PACKAGE_COPYRIGHT_YEAR in configure.ac.
 

-** Update NEWS
-The version number, *and* the date of the release (including for
-betas).
+** Update NEWS, commit and tag.
+See do-release-commit-and-tag in README-release.  For a while, we used
+beta names such as "2.6_rc1".  Now that we use gnulib in the release
+procedure, we must use "2.5.90", which has the additional benefit of
+being properly sorted in "git tag -l".

 
 

-** Update ChangeLog
-Should have an entry similar to `Version 1.49b.'.
+** make alpha, beta, or stable
+See README-release.

 
 

-** Tag the release
-Before Bison will build with the right version number, you must tag the release
-in git.  Do this after all other changes.  The command is similar to:
-
-  git tag -a v2.3b
+** Upload
+There are two ways to upload the tarballs to the GNU servers: using
+gnupload (from gnulib), or by hand.  Obviously prefer the former.  But
+in either case, be sure to read the following paragraph.

 
 

-The log message can be simply:
+*** Setup
+You need "gnupg".

 
 

-  Bison 2.3b
+Make sure your public key has been uploaded at least to
+keys.gnupg.net.  You can upload it with:

 
 

-** Push
-Once `make distcheck' passes, push your changes and the tag.
-`git push' without arguments will not push the tag.
+  gpg --keyserver keys.gnupg.net --send-keys F125BDF3

 
 

-** make alpha
-FIXME: `make alpha' is not maintained and is broken.  These
-instructions need to be replaced or removed.
+where F125BDF3 should be replaced with your key ID.

 
 

-Running `make alpha' is absolutely perfect for beta releases: it makes
-the tarballs, the xdeltas, and prepares (in /tmp/) a proto
-announcement.  It is so neat, that that's what I use anyway for
-genuine releases, but adjusting things by hand (e.g., the urls in the
-announcement file, the ChangeLog which is not needed etc.).
+*** Using gnupload
+You need "ncftp".

 
 

-If it fails, you're on your own...
+At the end "make stable" (or alpha/beta) will display the procedure to
+run.  Just copy and paste it in your shell.

 
 

-It requires GNU Make.
+*** By hand

 
 

-** Upload

 The generic GNU upload procedure is at:
 
   http://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads
 
 Follow the instructions there to register your information so you're permitted
 The generic GNU upload procedure is at:
 
   http://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads
 
 Follow the instructions there to register your information so you're permitted

-to upload.  Make sure your public key has been uploaded at least to
-keys.gnupg.net.  You can upload it with:
-
-  gpg --keyserver keys.gnupg.net --send-keys F125BDF3
-
-where F125BDF3 should be replaced with your key ID.
+to upload.

 
 Here's a brief reminder of how to roll the tarballs and upload them:
 
 *** make distcheck
 *** gpg -b bison-2.3b.tar.gz
 
 Here's a brief reminder of how to roll the tarballs and upload them:
 
 *** make distcheck
 *** gpg -b bison-2.3b.tar.gz

-*** In a file named `bison-2.3b.tar.gz.directive', type:
+*** In a file named "bison-2.3b.tar.gz.directive", type:

 
       version: 1.1
       directory: bison
 
       version: 1.1
       directory: bison


@@ -357,7 +408,8 @@ function 'index', once for the table of contents); you can ignore this
 issue.
 
 ** Announce
 issue.
 
 ** Announce

-To generate a template announcement file:
+The "make stable" (or alpha/beta) command just created a template,
+$HOME/announce-bison-X.Y.  Otherwise, to generate it, run:

 
   make RELEASE_TYPE=alpha gpg_key_ID=F125BDF3 announcement
 
 
   make RELEASE_TYPE=alpha gpg_key_ID=F125BDF3 announcement
 


@@ -367,28 +419,28 @@ replaced with your key ID.
 Complete/fix the announcement file.  The generated list of recipients
 (info-gnu@gnu.org, bug-bison@gnu.org, help-bison@gnu.org,
 bison-patches@gnu.org, and coordinator@translationproject.org) is
 Complete/fix the announcement file.  The generated list of recipients
 (info-gnu@gnu.org, bug-bison@gnu.org, help-bison@gnu.org,
 bison-patches@gnu.org, and coordinator@translationproject.org) is

-appropriate for a stable release or a ``serious beta''.  For any other
-release, drop at least info-gnu@gnu.org.  For an example of how to fill
-out the rest of the template, search the mailing list archives for the
-most recent release announcement.
+appropriate for a stable release or a "serious beta".  For any other
+release, drop at least info-gnu@gnu.org.  For an example of how to
+fill out the rest of the template, search the mailing list archives
+for the most recent release announcement.

 
 For a stable release, send the same announcement on the comp.compilers
 newsgroup by sending email to compilers@iecc.com.  Do not make any Cc as
 the moderator will throw away anything cross-posted or Cc'ed.  It really
 needs to be a separate message.
 
 
 For a stable release, send the same announcement on the comp.compilers
 newsgroup by sending email to compilers@iecc.com.  Do not make any Cc as
 the moderator will throw away anything cross-posted or Cc'ed.  It really
 needs to be a separate message.
 

-** Bump the version number
-In configure.ac.  Run `make'.  So that developers don't accidentally add new
-items to the old NEWS entry, create a new empty NEWS entry something like:
+** Prepare NEWS
+So that developers don't accidentally add new items to the old NEWS
+entry, create a new empty entry in line 3 (without the two leading
+spaces):

 
 

-  Changes in version ?.? (????-??-??):
+  * Noteworthy changes in release ?.? (????-??-??) [?]

 
 Push these changes.
 
 
 Push these changes.
 

-

 -----
 
 -----
 

-Copyright (C) 2002-2005, 2007-2011 Free Software Foundation, Inc.
+Copyright (C) 2002-2005, 2007-2013 Free Software Foundation, Inc.

 
 This file is part of GNU Bison.
 
 
 This file is part of GNU Bison.
 


@@ -404,3 +456,14 @@ GNU General Public License for more details.
 
 You should have received a copy of the GNU General Public License
 along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 You should have received a copy of the GNU General Public License
 along with this program.  If not, see <http://www.gnu.org/licenses/>.

+
+ LocalWords:  Automake Autoconf Gettext Gzip Rsync Valgrind gnulib submodules
+ LocalWords:  submodule init cd distcheck checkin ChangeLog valgrind sigreturn
+ LocalWords:  UC gcc DGNULIB POSIXCHECK xml XSLT glr lalr README po runtime rc
+ LocalWords:  gnupload gnupg gpg keyserver BDF ncftp filename clearsign cvs dir
+ LocalWords:  symlinks vti html lt POSIX Cc'ed
+
+Local Variables:
+mode: outline
+fill-column: 76
+End: