tools/tests/libMicro/AppleReadMe

   1 Mac OS X specific notes
   2
   3 *** Instructions before Starting libMicro ***
   4
   5 # Disable Open directory and LDAP using Directory Utility app
   6 # Turn off airport
   7 # Turn off spotlight. In terminal, execute the following:
   8   launchctl unload /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
   9 # Turn off Time Machine in System Preferences
  10 # Wait at least 2 minutes after boot to desktop for boot cache to settle down
  11
  12 *** Make and run quickstart ***
  13
  14         make
  15         ./bench >output.txt
  16 runs the libMicro test suite excluding the lmbench tests and gives you a text file named output.txt with the results of one run.
  17
  18
  19         ./multiview output1.txt output2.txt >compare.html
  20 gives you a html file comparing two runs.
  21
  22 *** To run libMicro testsuite with stepper disabled ***
  23 *** For Desktop use coreos_bench script***
  24
  25 To get a more consistent result of libMicro benchmark run, we need to disable the
  26 stepper to prevent it from causing wide variations in results. See rdar://6243819
  27 for details.
  28
  29 So to run libMicro test suite with stepper disabled, use 'coreos_bench' script
  30 instead of 'bench' script.
  31
  32 For example:
  33 ./coreos_bench > output.txt
  34 runs the libMicro test suite excluding the lmbench tests and gives you a text file named output.txt with the results of one run, with stepper disabled.
  35
  36 Note:
  37 1) We need '/usr/local/bin/pstates' to disable the stepper. Install AppleInternal package
  38 which provides '/usr/local/bin/pstates'.
  39
  40 2) 'coreos_bench' script is used exactly like the 'bench' script. All the usage examples for
  41 'bench' script in this readme file also holds true for 'coreos_bench' script.
  42
  43
  44
  45
  46
  47 *** Makefile ***
  48
  49 The Makefile invokes Makefile.Darwin which invokes Makefile.com.Darwin.
  50 Just invoke make, with options if necessary, and everything should
  51 build correctly. The binaries are placed in a directory called
  52 bin-ARCH where ARCH is the default or specified when building via
  53 the ARCH flag.
  54
  55 Note:
  56 1) The binaries of apple added tests are placed in a directory called
  57    apple/bin-ARCH
  58
  59 2) All the binaries under bin-ARCH and apple/bin-ARCH are code signed
  60    during build.
  61
  62 options for invoking Makefile are:
  63 ARCH            defaults to  i386
  64
  65         to build fat/multi architecture, specify
  66                 make ARCH=fat
  67         the makefile will automatically build with ARCH_FLAG="-arch i386 -arch x86_64" and put the results in bin-fat
  68
  69         to build for ARM architecture,
  70         first set an environment variable 'SDKROOT' to point to iPhone internal sdk
  71         For example:
  72                 $export SDKROOT="/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.Internal.sdk/"
  73         to build use:
  74                 make ARCH=ARM_ARCH where ARM_ARCH can be for e.g. armv6, armv7
  75         this will put the results in bin-ARM_ARCH
  76
  77         to build with only two of the architectures see below
  78
  79 ARCH_FLAG       defaults to -arch $(ARCH)
  80         to build fat/multi architecture, specify
  81                 make ARCH_FLAG="-arch i386" ARCH=fat
  82         this will put the results in bin-fat
  83
  84 OPT_FLAG        defaults to  -g
  85         to build optimized, specify make OPT_FLAG=-Os
  86
  87 SEMOP_FLAG      defaults to  -DUSE_SEMOP
  88         to eliminate SEMOP usage, specify make SEMOP_FLAG=
  89         this is needed on some lower-end systems (e.g. M63)
  90
  91 These can be combined, e.g.
  92         make ARCH=i386 SEMOP_FLAG=
  93
  94 *** Before running benchmarks ***
  95
  96 The shell script create_stuff should be run before any benchmarking
  97
  98 this script takes care of raising the process limits which would
  99 otherwise cause several of the tests to fail - if not you will see:
 100         Running:           pipe_pst1
 101         fork: Resource temporarily unavailable
 102 in your stderr during the runs. After you run create_stuff, the
 103 system then needs to be rebooted.
 104
 105 *** running the benchmarks ***
 106
 107 The shell script "bench" will run all the benchmarks, or you can
 108 pass it a parameter to run a single benchmark, e.g.
 109
 110 *** To run libMicro on the embedded platform, use "embd_bench" script. 'embd_bench' script is used exactly like the 'bench' script. All the usage examples for
 111 'bench' script in this readme file also holds true for 'embd_bench' script. ***
 112
 113         bench lmbench_bw_unix
 114
 115 By default the script will run only the libMicro testsuite excluding the lmbench tests.
 116 To run the libmicro testsuite with the lmbench tests included, just pass the -l parameter. e.g,
 117
 118         bench -l
 119 To run only the lmbench testsuite
 120
 121         bench lmbench
 122
 123 To display the usage, just do
 124         bench -h
 125
 126 Watch for:
 127         # WARNINGS
 128         #     Quantization error likely;increase batch size (-B option) 4X to avoid.
 129 in the output
 130 To see an example run the supplied testbench script
 131
 132 Add or adjust the -B parameter for any benchmark that fails. The
 133 Quantization error will refer to the benchmark preceding the error,
 134 not the one following...
 135
 136 A typical run:
 137         $ make clean
 138         $ make
 139         $ ./create_stuff
 140         $ ./bench > output1
 141         Running:              getpid
 142          for      0.13353 seconds
 143         Running:             getppid
 144          for      3.65609 seconds
 145         Running:              getenv
 146          for      0.20924 seconds
 147         Running:            getenvT2
 148          for      0.37437 seconds
 149         Running:        gettimeofday
 150          for      0.58077 seconds
 151         etc...
 152
 153 Use the supplied multiview script to compare runs like:
 154
 155 multiview output1 output2 > compare.html
 156 open compare.html (safari launches)
 157 will show output2 results as a percentage change from the output1 results
 158
 159 *** Adding additional benchmark tests ***
 160
 161 Look at the sample file trivial.c.  This demonstrates how to do
 162 argument passing, the flow of control of a benchmark, etc. for the
 163 trivial case.  The tests starting with "lmbench_" were ported from
 164 the lmbench suite, so they might be good examples as well.
 165
 166 *** A note regarding future changes in bench.sh script ***
 167 coreos_bench.sh script is almost identical to bench.sh script, except that it
 168 has additional code to disable the stepper during libmicro benchmark run.
 169
 170 In future, if bench.sh script is modified, make sure the changes reflect
 171 in coreos_bench.sh script also.
 172
 173 *** Things to do ***
 174
 175 * port the rest of the lmbench benchmarks into this framework
 176
 177 * create website that will allow easy ability to compare many builds
 178 across many machines with historical repository of runs
 179
 180 * document better how to write a benchmark for this framework
 181 (started in trivial.c)
 182
 183 * check this into xnu/test
 184
 185 * create new benchmarks
 186
 187 *** Leopard notes ***
 188
 189 Due to rdar://4654956 and its original, rdar://2588252 you cannot
 190 run these tests on Leopard without removing the cascade_lockf test.
 191 There may be other tests which panic a Leopard system.
 192
 193 *** benchDS notes ***
 194
 195 From rdar://problem/7468995 add the ability to benchmark the key APIs
 196 for server daemons.  In particular, a test binary is added for each of:
 197
 198         ODQueryCreateWithNode()  (standard User, Groups, and Hosts records)
 199         getaddrinfo()  (hosts and ports)
 200         mbr_check_service_membership()
 201         mbr_check_membership()
 202         getpwnam()
 203         getpwuid()
 204         getgrgid()
 205         getpwent()
 206         getgrent()
 207         getgrnam()
 208
 209 The script benchDS is provided to run a standard set of tests presuming
 210 that the tests are run by root on a system configured with an OD binding.
 211 The OD server (local or remote) must have a set of accounts created with
 212 od_acount_create shell script.  This script must also be run as root,
 213 and passed a single argument of the number of users to create.  It creates
 214 od_test_{1..N}, and all belong to a ds_test_group1(gid 1211).  In addition,
 215 ds_test_group2(gid 1212) is created which has no users as members.  User ids are
 216 set sequentially from 5000.  In order to administer the OD server, it assumes
 217 user 'diradmin' and password 'admin' are the OD admin.
 218
 219 Also, these tests consult the APIs listed, which can be run against the local
 220 account info, or even Active Directory.
 221
 222 Thus, the quick recipe is:
 223         Install X Server
 224         Enable OD, and create directory admin user 'diradmin' with password 'admin'
 225         As root run:  od_account_create 1000
 226         Now run the test, as root:  ./benchDS 1000 > output-file
 227
 228
 229 In addition, od_account_delete 1000 will delete the 1000 users created with od_account_create.
 230
 231