tools/tests/xnu_quick_test/README

   1 xnu_quick_test - this tool will do a quick test of every (well, to be
   2 honest most) system calls we support in xnu.
   3
   4 WARNING - this is not meant to be a full regression test of all the
   5 system calls.  The intent is to have a quick test of each system call that
   6 can be run very easily and quickly when a new kernel is built.
   7
   8 This tool is meant to grow as we find xnu problems that could have be
   9 caught before we submit to a build train.  So please add more tests and
  10 make the existing ones better.  Some of the original tests are nothing
  11 more than place holders and quite lame.  Just keep in mind that the tool
  12 should run as fast as possible.  If it gets too slow then most people
  13 will stop running it.
  14
  15 LP64 testing tip - when adding or modifying tests, keep in mind the
  16 variants in the LP64 world.  If xnu gets passed a structure the varies in
  17 size between 32 and 64-bit processes, try to test that a field in the
  18 structure contains valid data.  For example, if we know foo structure
  19 looks like:
  20 struct foo {
  21              int             an_int;
  22              long    a_long;
  23              int             another_int;
  24 }
  25 And if we know what another_int should contain then test for the known
  26 value since it's offset will vary depending on whether the calling process
  27 is 32 or 64 bits.
  28
  29 NOTE - we have several workarounds and test exceptions for some
  30 outstanding bugs in xnu.  All the workarounds are marked with "todo" and
  31 some comments noting the radar number of the offending bug.  Do a seach
  32 for "todo" in the source files for this project to locate which tests have
  33 known failures.   And please tag any new exceptions you find with "todo"
  34 in the comment and the radar number of the bug.
  35
  36 To build a fat binary, export ARCH="ppc ppc64 i386 x86_64". This will work
  37 for any architectures that Apple gcc recognizes.
  38
  39 Added three defines which you can use at the compile line to build variants.
  40 DEBUG
  41        turn on additional printfs
  42 CONFORMANCE_TESTS_IN_XNU
  43        when conformance tests are in xnu, set this to 1
  44 TEST_SYSTEM_CALLS
  45        test system calls (doesn't compile; a different bug)
  46 by default, all three are set to 0, i.e. disabled.  To build, export
  47 MORECFLAGS with the values you want set, e.g.
  48        export MORECFLAGS="-D DEBUG=1 -D CONFORMANCE_TESTS_IN_XNU=1"
  49
  50 todo:
  51 -- have folks with area expertise fix lame tests
  52    (most of the networking related tests are pretty lame)
  53 -- mach system calls support
  54
  55
  56 USAGE:  xnu_quick_test -target TARGET_PATH
  57
  58          -f[ailures] MAX_FAILS_ALLOWED   # number of test cases that may fail before we give up.  defaults to 0
  59          -l[ist]                         # list all the tests this tool performs
  60          -r[un] 1, 3, 10 - 19            # run specific tests.  enter individual test numbers and/or range of numbers.  use -list to list tests.
  61          -s[kip]                         # skip setuid tests
  62          -t[arget] TARGET_PATH           # path to directory where tool will create test files.  defaults to "/tmp/"
  63
  64 examples:
  65 --- Place all test files and directories at the root of volume "test_vol" ---
  66 xnu_quick_test -t /Volumes/test_vol/
  67
  68 --- Run the tool for tests 10 thru 15, test 18 and test 20 ---
  69 xnu_quick_test -r 10-15, 18, 20
  70
  71
  72 --- example of running the tool to list all the tests it currently supports ---
  73 xnu_quick_test -l
  74 List of all tests this tool performs...
  75  1         syscall
  76  2         fork, wait4, exit
  77  3         fsync, ftruncate, lseek, pread, pwrite, read, readv, truncate, write, writev
  78  4         close, fpathconf, fstat, open, pathconf
  79  5         link, stat, unlink
  80  6         chdir, fchdir
  81  7         access, chmod, fchmod
  82  8         chown, fchown, lchown, lstat, readlink, symlink
  83  9         fstatfs, getattrlist, getfsstat, statfs
  84  10        getpid, getppid, pipe
  85  11        getauid, gettid, getuid, geteuid, issetugid, setaudit_addr, seteuid, settid, settid_with_pid, setuid
  86  12        mkdir, rmdir, umask
  87  13        mknod, sync
  88  14        fsync, getsockopt, poll, select, setsockopt, socketpair
  89  15        accept, bind, connect, getpeername, getsockname, listen, socket, recvmsg, sendmsg, sendto, sendfile
  90  16        chflags, fchflags
  91  17        kill, vfork, execve
  92  18        getegid, getgid, getgroups, setegid, setgid, setgroups
  93  19        dup, dup2, getdtablesize
  94  20        getrusage, profil
  95  21        getitimer, setitimer, sigaction, sigpending, sigprocmask, sigsuspend, sigwait
  96  22        acct
  97  23        ioctl
  98  24        chroot
  99  25        madvise, mincore, minherit, mlock, mlock, mmap, mprotect, msync, munmap
 100  26        getpgrp, getpgid, getsid, setpgid, setpgrp, setsid
 101  27        fcntl
 102  28        getlogin, setlogin
 103  29        getpriority, setpriority
 104  30        futimes, gettimeofday, settimeofday, utimes
 105  31        rename, stat
 106  32        flock
 107  33        mkfifo, read, write
 108  34        quotactl
 109  35        getrlimit, setrlimit
 110  36        getattrlist, getdirentries, getdirentriesattr, setattrlist
 111  37        exchangedata
 112  38        searchfs
 113  39        sem_close, sem_open, sem_post, sem_trywait, sem_unlink, sem_wait
 114  40        semctl, semget, semop
 115  41        shm_open, shm_unlink
 116  42        shmat, shmctl, shmdt, shmget
 117  43        fgetxattr, flistxattr, fremovexattr, fsetxattr, getxattr, listxattr, removexattr, setxattr
 118  44        aio_cancel, aio_error, aio_read, aio_return, aio_suspend, aio_write, fcntl, lio_listio
 119  45        kevent, kqueue
 120  46        msgctl, msgget, msgrcv, msgsnd
 121
 122
 123 --- example of a full test run ---
 124 cottje% ./BUILD/dst/xnu_quick_test
 125 created test directory at "/tmp/xnu_quick_test-1660251855"
 126 Will allow 0 failures before testing is aborted
 127
 128 Begin testing - Tue Dec 13 15:56:50 2005
 129
 130 test #1 - syscall
 131 test #2 - fork, wait4, exit
 132 test #3 - fsync, ftruncate, lseek, pread, pwrite, read, readv, truncate, write, writev
 133 test #4 - close, fpathconf, fstat, open, pathconf
 134 test #5 - link, stat, unlink
 135 test #6 - chdir, fchdir
 136 test #7 - access, chmod, fchmod
 137 test #8 - chown, fchown, lchown, lstat, readlink, symlink
 138 test #9 - fstatfs, getattrlist, getfsstat, statfs
 139 test #10 - getpid, getppid, pipe
 140 test #11 - getauid, gettid, getuid, geteuid, issetugid, setauid, seteuid, settid, settid_with_pid, setuid
 141 test #12 - mkdir, rmdir, umask
 142 test #13 - mknod, sync
 143 test #14 - fsync, getsockopt, poll, select, setsockopt, socketpair
 144 test #15 - accept, bind, connect, getpeername, getsockname, listen, socket, recvmsg, sendmsg, sendto
 145 test #16 - chflags, fchflags
 146 test #17 - kill, vfork, execve
 147 test #18 - getegid, getgid, getgroups, setegid, setgid, setgroups
 148 test #19 - dup, dup2, getdtablesize
 149 test #20 - getrusage, profil
 150 test #21 - ktrace
 151 test #22 - getitimer, setitimer, sigaction, sigpending, sigprocmask, sigsuspend, sigwait
 152 test #23 - acct
 153 test #24 - ioctl
 154 test #25 - chroot
 155 test #26 - madvise, mincore, minherit, mlock, mlock, mmap, mprotect, msync, munmap
 156 test #27 - getpgrp, getpgid, getsid, setpgid, setpgrp, setsid
 157 test #28 - fcntl
 158 test #29 - getlogin, setlogin
 159 test #30 - getpriority, setpriority
 160 test #31 - futimes, gettimeofday, settimeofday, utimes
 161 test #32 - rename, stat
 162 test #33 - flock
 163 test #34 - mkfifo, read, write
 164 test #35 - quotactl
 165 test #36 - getrlimit, setrlimit
 166 test #37 - getattrlist, getdirentries, getdirentriesattr, setattrlist
 167 test #38 - exchangedata
 168 test #39 - searchfs
 169 test #40 - sem_close, sem_open, sem_post, sem_trywait, sem_unlink, sem_wait
 170 test #41 - semctl, semget, semop
 171 test #42 - shm_open, shm_unlink
 172 test #43 - shmat, shmctl, shmdt, shmget
 173 test #44 - fgetxattr, flistxattr, fremovexattr, fsetxattr, getxattr, listxattr, removexattr, setxattr
 174 test #45 - aio_cancel, aio_error, aio_read, aio_return, aio_suspend, aio_write, fcntl, lio_listio
 175 test #46 - kevent, kqueue
 176 test #47 - msgctl, msgget, msgrcv, msgsnd
 177
 178 End testing - Tue Dec 13 15:57:08 2005