tools/tests/libMicro/AppleReadMe

   1 Mac OS X specific notes
   2
   3 *** Instructions before Starting libMicro ***
   4
   5 # Disable Open directory and LDAP using Directory Utility app
   6 # Turn off airport
   7 # Turn off spotlight. In terminal, execute the following:
   8   launchctl unload /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
   9 # Turn off Time Machine in System Preferences
  10 # Wait at least 2 minutes after boot to desktop for boot cache to settle down
  11
  12 *** Make and run quickstart ***
  13
  14         make
  15         ./bench >output.txt
  16 runs the libMicro test suite excluding the lmbench tests and gives you a text file named output.txt with the results of one run.
  17
  18
  19         ./multiview output1.txt output2.txt >compare.html
  20 gives you a html file comparing two runs.
  21
  22 *** To run libMicro testsuite with stepper disabled ***
  23
  24 To get a more consistent result of libMicro benchmark run, we need to disable the
  25 stepper to prevent it from causing wide variations in results. See rdar://6243819
  26 for details.
  27
  28 So to run libMicro test suite with stepper disabled, use 'coreos_bench' script
  29 instead of 'bench' script.
  30
  31 For example:
  32 ./coreos_bench > output.txt
  33 runs the libMicro test suite excluding the lmbench tests and gives you a text file named output.txt with the results of one run, with stepper disabled.
  34
  35 Note:
  36 1) We need '/usr/local/bin/pstates' to disable the stepper. Install AppleInternal package
  37 which provides '/usr/local/bin/pstates'.
  38
  39 2) 'coreos_bench' script is used exactly like the 'bench' script. All the usage examples for
  40 'bench' script in this readme file also holds true for 'coreos_bench' script.
  41
  42 *** Makefile ***
  43
  44 The Makefile invokes Makefile.Darwin which invokes Makefile.com.Darwin.
  45 Just invoke make, with options if necessary, and everything should
  46 build correctly. The binaries are placed in a directory called
  47 bin-ARCH where ARCH is the default or specified when building via
  48 the ARCH flag.
  49
  50 Note:
  51 1) The binaries of apple added tests are placed in a directory called
  52    apple/bin-ARCH
  53
  54 2) All the binaries under bin-ARCH and apple/bin-ARCH are code signed
  55    during build.
  56
  57 options for invoking Makefile are:
  58 ARCH            defaults to  i386
  59
  60         to build fat/multi architecture, specify
  61                 make ARCH=fat
  62         the makefile will automatically build with ARCH_FLAG="-arch i386 -arch x86_64" and put the results in bin-fat
  63
  64         to build for ARM architecture,
  65         first set an environment variable 'SDKROOT' to point to iPhone sdk
  66                 make ARCH=ARM_ARCH where ARM_ARCH can be armv6 or armv7
  67         this will put the results in bin-ARM_ARCH
  68
  69         to build with only two of the architectures see below
  70
  71 ARCH_FLAG       defaults to -arch $(ARCH)
  72         to build fat/multi architecture, specify
  73                 make ARCH_FLAG="-arch i386" ARCH=fat
  74         this will put the results in bin-fat
  75
  76 OPT_FLAG        defaults to  -g
  77         to build optimized, specify make OPT_FLAG=-Os
  78
  79 SEMOP_FLAG      defaults to  -DUSE_SEMOP
  80         to eliminate SEMOP usage, specify make SEMOP_FLAG=
  81         this is needed on some lower-end systems (e.g. M63)
  82
  83 These can be combined, e.g.
  84         make ARCH=i386 SEMOP_FLAG=
  85
  86 *** Before running benchmarks ***
  87
  88 The shell script create_stuff should be run before any benchmarking
  89
  90 this script takes care of raising the process limits which would
  91 otherwise cause several of the tests to fail - if not you will see:
  92         Running:           pipe_pst1
  93         fork: Resource temporarily unavailable
  94 in your stderr during the runs. After you run create_stuff, the
  95 system then needs to be rebooted.
  96
  97 *** running the benchmarks ***
  98
  99 The shell script "bench" will run all the benchmarks, or you can
 100 pass it a parameter to run a single benchmark, e.g.
 101
 102         bench lmbench_bw_unix
 103
 104 By default the script will run only the libMicro testsuite excluding the lmbench tests.
 105 To run the libmicro testsuite with the lmbench tests included, just pass the -l parameter. e.g,
 106
 107         bench -l
 108 To run only the lmbench testsuite
 109
 110         bench lmbench
 111
 112 To display the usage, just do
 113         bench -h
 114
 115 Watch for:
 116         # WARNINGS
 117         #     Quantization error likely;increase batch size (-B option) 4X to avoid.
 118 in the output
 119 To see an example run the supplied testbench script
 120
 121 Add or adjust the -B parameter for any benchmark that fails. The
 122 Quantization error will refer to the benchmark preceding the error,
 123 not the one following...
 124
 125 A typical run:
 126         $ make clean
 127         $ make
 128         $ ./create_stuff
 129         $ ./bench > output1
 130         Running:              getpid
 131          for      0.13353 seconds
 132         Running:             getppid
 133          for      3.65609 seconds
 134         Running:              getenv
 135          for      0.20924 seconds
 136         Running:            getenvT2
 137          for      0.37437 seconds
 138         Running:        gettimeofday
 139          for      0.58077 seconds
 140         etc...
 141
 142 Use the supplied multiview script to compare runs like:
 143
 144 multiview output1 output2 > compare.html
 145 open compare.html (safari launches)
 146 will show output2 results as a percentage change from the output1 results
 147
 148 *** Adding additional benchmark tests ***
 149
 150 Look at the sample file trivial.c.  This demonstrates how to do
 151 argument passing, the flow of control of a benchmark, etc. for the
 152 trivial case.  The tests starting with "lmbench_" were ported from
 153 the lmbench suite, so they might be good examples as well.
 154
 155 *** A note regarding future changes in bench.sh script ***
 156 coreos_bench.sh script is almost identical to bench.sh script, except that it
 157 has additional code to disable the stepper during libmicro benchmark run.
 158
 159 In future, if bench.sh script is modified, make sure the changes reflect
 160 in coreos_bench.sh script also.
 161
 162 *** Things to do ***
 163
 164 * port the rest of the lmbench benchmarks into this framework
 165
 166 * create website that will allow easy ability to compare many builds
 167 across many machines with historical repository of runs
 168
 169 * document better how to write a benchmark for this framework
 170 (started in trivial.c)
 171
 172 * check this into xnu/test
 173
 174 * create new benchmarks
 175
 176 *** Leopard notes ***
 177
 178 Due to rdar://4654956 and its original, rdar://2588252 you cannot
 179 run these tests on Leopard without removing the cascade_lockf test.
 180 There may be other tests which panic a Leopard system.
 181
 182 *** benchDS notes ***
 183
 184 From rdar://problem/7468995 add the ability to benchmark the key APIs
 185 for server daemons.  In particular, a test binary is added for each of:
 186
 187         ODQueryCreateWithNode()  (standard User, Groups, and Hosts records)
 188         getaddrinfo()  (hosts and ports)
 189         mbr_check_service_membership()
 190         mbr_check_membership()
 191         getpwnam()
 192         getpwuid()
 193         getgrgid()
 194         getpwent()
 195         getgrent()
 196         getgrnam()
 197
 198 The script benchDS is provided to run a standard set of tests presuming
 199 that the tests are run by root on a system configured with an OD binding.
 200 The OD server (local or remote) must have a set of accounts created with
 201 od_acount_create shell script.  This script must also be run as root,
 202 and passed a single argument of the number of users to create.  It creates
 203 od_test_{1..N}, and all belong to a ds_test_group1(gid 1211).  In addition,
 204 ds_test_group2(gid 1212) is created which has no users as members.  User ids are
 205 set sequentially from 5000.  In order to administer the OD server, it assumes
 206 user 'diradmin' and password 'admin' are the OD admin.
 207
 208 Also, these tests consult the APIs listed, which can be run against the local
 209 account info, or even Active Directory.
 210
 211 Thus, the quick recipe is:
 212         Install X Server
 213         Enable OD, and create directory admin user 'diradmin' with password 'admin'
 214         As root run:  od_account_create 1000
 215         Now run the test, as root:  ./benchDS 1000 > output-file
 216
 217
 218 In addition, od_account_delete 1000 will delete the 1000 users created with od_account_create.
 219
 220