[apple/xnu.git] / tools / tests / libMicro / AppleReadMe

Mac OS X specific notes

*** Instructions before Starting libMicro ***

# Disable Open directory and LDAP using Directory Utility app
# Turn off airport
# Turn off spotlight. In terminal, execute the following:
  sudo service com.apple.metadata.mds stop
# Turn off Time Machine in System Preferences
# Wait at least 2 minutes after boot to desktop for boot cache to settle down
   
*** Make and run quickstart ***

	make
	./bench >output.txt
gives you a text file named output.txt with the results of one run.
	./multiview output1.txt output2.txt >compare.html
gives you a html file comparing two runs.

*** Makefile ***

The Makefile invokes Makefile.Darwin which invokes Makefile.com.Darwin.
Just invoke make, with options if necessary, and everything should
build correctly. The binaries are placed in a directory called
bin-ARCH where ARCH is the default or specified when building via
the ARCH flag.

options for invoking Makefile are:
ARCH 		defaults to  i386
	if you just want to build for ppc, you can specify
		make ARCH=ppc
	this will put the results in bin-ppc

        to build fat/multi architecture, specify 
		make ARCH=fat
	the makefile will automatically build with ARCH_FLAG="-arch ppc -arch i386 -arch x86_64" and put the results in bin-fat

	to build with only two of the architectures see below

ARCH_FLAG 	defaults to -arch $(ARCH)
	to build fat/multi architecture, specify 
		make ARCH_FLAG="-arch ppc -arch i386" ARCH=fat
	this will put the results in bin-fat

OPT_FLAG 	defaults to  -g
	to build optimized, specify make OPT_FLAG=-s

SEMOP_FLAG 	defaults to  -DUSE_SEMOP
	to eliminate SEMOP usage, specify make SEMOP_FLAG=
	this is needed on some lower-end systems (e.g. M63)

These can be combined, e.g.
	make ARCH=ppc SEMOP_FLAG=

*** Before running benchmarks ***

The shell script create_stuff should be run before any benchmarking

this script takes care of raising the process limits which would
otherwise cause several of the tests to fail - if not you will see:
	Running:           pipe_pst1
	fork: Resource temporarily unavailable
in your stderr during the runs. After you run create_stuff, the
system then needs to be rebooted.

*** running the benchmarks ***

The shell script "bench" will run all the benchmarks, or you can
pass it a parameter to run a single benchmark, e.g.

	bench lmbench_bw_unix

Watch for:
	# WARNINGS
	#     Quantization error likely;increase batch size (-B option) 4X to avoid.
in the output
To see an example run the supplied testbench script

Add or adjust the -B parameter for any benchmark that fails. The
Quantization error will refer to the benchmark preceding the error,
not the one following...

A typical run:
	$ make clean
	$ make
	$ ./create_stuff
	$ ./bench > output1
	Running:              getpid
	 for      0.13353 seconds
	Running:             getppid
	 for      3.65609 seconds
	Running:              getenv
	 for      0.20924 seconds
	Running:            getenvT2
	 for      0.37437 seconds
	Running:        gettimeofday
	 for      0.58077 seconds
	etc...

Use the supplied multiview script to compare runs like:

multiview output1 output2 > compare.html
open compare.html (safari launches)
will show output2 results as a percentage change from the output1 results

*** Adding additional benchmark tests ***

Look at the sample file trivial.c.  This demonstrates how to do
argument passing, the flow of control of a benchmark, etc. for the
trivial case.  The tests starting with "lmbench_" were ported from
the lmbench suite, so they might be good examples as well.

*** Things to do ***

* port the rest of the lmbench benchmarks into this framework

* create website that will allow easy ability to compare many builds
across many machines with historical repository of runs

* document better how to write a benchmark for this framework
(started in trivial.c)

* check this into xnu/test

* create new benchmarks

*** Leopard notes ***

Due to rdar://4654956 and its original, rdar://2588252 you cannot
run these tests on Leopard without removing the cascade_lockf test.
There may be other tests which panic a Leopard system.
Commit	Line	Data
b0d623f7 A	1	Mac OS X specific notes
	2
	3	* Instructions before Starting libMicro *
	4
	5	# Disable Open directory and LDAP using Directory Utility app
	6	# Turn off airport
	7	# Turn off spotlight. In terminal, execute the following:
	8	sudo service com.apple.metadata.mds stop
	9	# Turn off Time Machine in System Preferences
	10	# Wait at least 2 minutes after boot to desktop for boot cache to settle down
	11
	12	* Make and run quickstart *
	13
	14	make
	15	./bench >output.txt
	16	gives you a text file named output.txt with the results of one run.
	17	./multiview output1.txt output2.txt >compare.html
	18	gives you a html file comparing two runs.
	19
	20	* Makefile *
	21
	22	The Makefile invokes Makefile.Darwin which invokes Makefile.com.Darwin.
	23	Just invoke make, with options if necessary, and everything should
	24	build correctly. The binaries are placed in a directory called
	25	bin-ARCH where ARCH is the default or specified when building via
	26	the ARCH flag.
	27
	28	options for invoking Makefile are:
	29	ARCH defaults to i386
	30	if you just want to build for ppc, you can specify
	31	make ARCH=ppc
	32	this will put the results in bin-ppc
	33
	34	to build fat/multi architecture, specify
	35	make ARCH=fat
	36	the makefile will automatically build with ARCH_FLAG="-arch ppc -arch i386 -arch x86_64" and put the results in bin-fat
	37
	38	to build with only two of the architectures see below
	39
	40	ARCH_FLAG defaults to -arch $(ARCH)
	41	to build fat/multi architecture, specify
	42	make ARCH_FLAG="-arch ppc -arch i386" ARCH=fat
	43	this will put the results in bin-fat
	44
	45	OPT_FLAG defaults to -g
	46	to build optimized, specify make OPT_FLAG=-s
	47
	48	SEMOP_FLAG defaults to -DUSE_SEMOP
	49	to eliminate SEMOP usage, specify make SEMOP_FLAG=
	50	this is needed on some lower-end systems (e.g. M63)
	51
	52	These can be combined, e.g.
	53	make ARCH=ppc SEMOP_FLAG=
	54
	55	* Before running benchmarks *
	56
	57	The shell script create_stuff should be run before any benchmarking
	58
	59	this script takes care of raising the process limits which would
	60	otherwise cause several of the tests to fail - if not you will see:
	61	Running: pipe_pst1
	62	fork: Resource temporarily unavailable
	63	in your stderr during the runs. After you run create_stuff, the
	64	system then needs to be rebooted.
65
66	* running the benchmarks *
67
68	The shell script "bench" will run all the benchmarks, or you can
69	pass it a parameter to run a single benchmark, e.g.
70
71	bench lmbench_bw_unix
72
73	Watch for:
74	# WARNINGS
75	# Quantization error likely;increase batch size (-B option) 4X to avoid.
76	in the output
77	To see an example run the supplied testbench script
78
79	Add or adjust the -B parameter for any benchmark that fails. The
80	Quantization error will refer to the benchmark preceding the error,
81	not the one following...
82
83	A typical run:
84	$ make clean
85	$ make
86	$ ./create_stuff
87	$ ./bench > output1
88	Running: getpid
89	for 0.13353 seconds
90	Running: getppid
91	for 3.65609 seconds
92	Running: getenv
93	for 0.20924 seconds
94	Running: getenvT2
95	for 0.37437 seconds
96	Running: gettimeofday
97	for 0.58077 seconds
98	etc...
99
100	Use the supplied multiview script to compare runs like:
101
102	multiview output1 output2 > compare.html
103	open compare.html (safari launches)
104	will show output2 results as a percentage change from the output1 results
105
106	* Adding additional benchmark tests *
107
108	Look at the sample file trivial.c. This demonstrates how to do
109	argument passing, the flow of control of a benchmark, etc. for the
110	trivial case. The tests starting with "lmbench_" were ported from
111	the lmbench suite, so they might be good examples as well.
112
113	* Things to do *
114
115	* port the rest of the lmbench benchmarks into this framework
116
117	* create website that will allow easy ability to compare many builds
118	across many machines with historical repository of runs
119
120	* document better how to write a benchmark for this framework
121	(started in trivial.c)
122
123	* check this into xnu/test
124
125	* create new benchmarks
126
127	* Leopard notes *
128
129	Due to rdar://4654956 and its original, rdar://2588252 you cannot
130	run these tests on Leopard without removing the cascade_lockf test.
131	There may be other tests which panic a Leopard system.