[apple/icu.git] / icuSources / test / perf / README

Copyright (C) 2016 and later: Unicode, Inc. and others.
License & terms of use: http://www.unicode.org/copyright.html#License

Copyright (C) 2008-2013, International Business Machines
Corporation and others.  All Rights Reserved.

README for ICU4C Performance Test

Introduction:
The performance tests in ICU4C compares various aspects of ICU against previous versions of ICU,
Posix, and/or Windows.  Some tests only run on Windows and are labeled accordingly.All of the 
performance tests are driven by a perl-based script which calls the underlying C program and 
displays the statistical analysis of the test in an easy to read HTML web page.  Each test will
have its own web page.


Requirements:
The most up to date tests are in ICU 4.0 and later. In addition, most tests depend on data which are 
in a separate repository.

Data repository:
http://source.icu-project.org/repos/icu/data/trunk/test/perf
There are 3 folders in here: collation, conversion, and udr.


Running Performance Test:

1) Checkout the current ICU and 2 previous versions (for proper regression testing) and the data
   into your local system.

2) Do a general build on all the ICUs either through Visual Studios or with the Makefile.

3) Build the ICU performance tests either through Visual Studios or with the Makefile.
   (The location of the performance test is: icu/source/test/perf)

4) Go to perf/perldriver, copy Common.pl.template to Common.pl.
   In this file, you will need to set a few variables:
   a) ICU version numbers
   b) Path to the performance test data
   c) Path to the root directory of the different versions of ICU
   d) Whether or not you are running the test on Windows
   * The Path needs to be absolute
   * All other variables should be left as is

5) Create a directory pref/results. All of the test result web pages will be
   stored in this directory.

6) In the perf directory, there are several other directories for the various tests that can be run.
   (e.g. convperf (Conversion tests)). Go to each directory and execute the perl script for each test
   you want to run. (e.g. CharPerf_r.pl in CharPerf directory)
   * You only need to run the perl script from the latest ICU version.  The corresponding perl script 
     in the previous versions of ICU do not need to be run seperately.


Note: When running on Windows, cygwin is preferred.
Note: If a test needs to be run on Windows, a message will be displayed.
Note: After building the performance tests, it is a good idea to execute each C program to ensure that each
      test can run. (e.g. convperf.exe)
Note: To run the actual performance test, you will need to setup Perl with the following modules:
      a) Statistics/Distribution.pm
      b) Statistics/Descriptive.pm 
Note: Some tests take a while depending on the machine it is running on so a script to run each performance test's
      perl script might be helpful.
Note: The perl script is only used in one version of ICU. When you run regression tests,
      it is recommended to run the tests from the later version of ICU.
Commit	Line	Data
f3c0d7a5 A	1	Copyright (C) 2016 and later: Unicode, Inc. and others.
	2	License & terms of use: http://www.unicode.org/copyright.html#License
	3
57a6839d	4	Copyright (C) 2008-2013, International Business Machines
46f4442e A	5	Corporation and others. All Rights Reserved.
	6
	7	README for ICU4C Performance Test
	8
	9	Introduction:
	10	The performance tests in ICU4C compares various aspects of ICU against previous versions of ICU,
	11	Posix, and/or Windows. Some tests only run on Windows and are labeled accordingly.All of the
	12	performance tests are driven by a perl-based script which calls the underlying C program and
	13	displays the statistical analysis of the test in an easy to read HTML web page. Each test will
	14	have its own web page.
	15
	16
	17	Requirements:
	18	The most up to date tests are in ICU 4.0 and later. In addition, most tests depend on data which are
57a6839d	19	in a separate repository.
46f4442e A	20
	21	Data repository:
	22	http://source.icu-project.org/repos/icu/data/trunk/test/perf
	23	There are 3 folders in here: collation, conversion, and udr.
	24
	25
	26	Running Performance Test:
	27
57a6839d A	28	1) Checkout the current ICU and 2 previous versions (for proper regression testing) and the data
57a6839d A	29	into your local system.
46f4442e	30
57a6839d	31	2) Do a general build on all the ICUs either through Visual Studios or with the Makefile.
46f4442e	32
57a6839d	33	3) Build the ICU performance tests either through Visual Studios or with the Makefile.
46f4442e A	34	(The location of the performance test is: icu/source/test/perf)
46f4442e A	35
57a6839d	36	4) Go to perf/perldriver, copy Common.pl.template to Common.pl.
46f4442e A	37	In this file, you will need to set a few variables:
	38	a) ICU version numbers
	39	b) Path to the performance test data
	40	c) Path to the root directory of the different versions of ICU
	41	d) Whether or not you are running the test on Windows
	42	* The Path needs to be absolute
	43	* All other variables should be left as is
	44
57a6839d A	45	5) Create a directory pref/results. All of the test result web pages will be
	46	stored in this directory.
	47
46f4442e A	48	6) In the perf directory, there are several other directories for the various tests that can be run.
	49	(e.g. convperf (Conversion tests)). Go to each directory and execute the perl script for each test
	50	you want to run. (e.g. CharPerf_r.pl in CharPerf directory)
	51	* You only need to run the perl script from the latest ICU version. The corresponding perl script
	52	in the previous versions of ICU do not need to be run seperately.
46f4442e A	53
	54
	55	Note: When running on Windows, cygwin is preferred.
	56	Note: If a test needs to be run on Windows, a message will be displayed.
	57	Note: After building the performance tests, it is a good idea to execute each C program to ensure that each
	58	test can run. (e.g. convperf.exe)
	59	Note: To run the actual performance test, you will need to setup Perl with the following modules:
	60	a) Statistics/Distribution.pm
	61	b) Statistics/Descriptive.pm
	62	Note: Some tests take a while depending on the machine it is running on so a script to run each performance test's
	63	perl script might be helpful.
57a6839d A	64	Note: The perl script is only used in one version of ICU. When you run regression tests,
57a6839d A	65	it is recommended to run the tests from the later version of ICU.