SecurityTests/cspxutils/README

   1                          cspxutils Info
   2                                    last update 9/12/06 dmitch
   3
   4 This directory contains a set of programs intended for use in testing the
   5 AppleCSP on the OS X platform. The intended users are software engineers
   6 who are familiar with the CSSM CSP API. The programs are used for design
   7 verification testing (DVT), for interactive exercising of the CSP
   8 subsystem during bringup, and for subsequent regression testing.
   9
  10 The intention is that there is a test here for every supported CSP function.
  11 No additions to the CSP functionality will be checked into CVS unless there
  12 is a corresponding test here to verify that functionality.
  13
  14
  15 Building
  16 --------
  17
  18 Short version: to build everything in cspxutils, cd to cspxutils and
  19 type 'make'. The resulting executables are placed in the location specified
  20 in the LOCAL_BUILD_DIR environment variable (which you really must set to use
  21 these tests effectively).
  22
  23 There is also an 'all' target in the top level Makefile which builds not
  24 only everything here, but everything in ../clxutils as well. This target
  25 requires the LOCAL_BUILD_DIR environment variable to work.
  26
  27 Long version: The programs in cspxutils build via standard Unix
  28 Makefiles, not by Xcode. You generally want to set LOCAL_BUILD_DIR to
  29 the same place Xcode places its built binaries ("Place Build Products in:"
  30 in the Building pane in Xcode app's Preferences). This package builds OK
  31 if you do not have $LOCAL_BUILD_DIR set, but the installed
  32 Security.framework has to be up-to-date, and the cspdvt script (see
  33 below) will not run since executables are placed in their respective
  34 source directories.
  35
  36 (Note: the number of times the LOCAL_BUILD_DIR environment variable is
  37 mentioned here tends to lead an alert user of this code to just set that in
  38 the user's .cshrc or equivalent and be done with it.)
  39
  40 The source for each program is contained in its own directory; you can
  41 build a program on its own by typing 'make' in that directory. There is
  42 also a top-level Makefile in cspxutils so that you can go there and type
  43 'make' and all of the included programs will build. The 'make clean'
  44 target is supported at both levels.
  45
  46 This source tree is generally up-to-date with top-of-tree tla/.
  47 No changes will be checked in to TOT cspxutils until the corresponding
  48 changes to Security, if any are needed, are also checked into TOT
  49 Security.
  50
  51
  52 Running
  53 -------
  54
  55 Each executable can be run on its own, generally from anywhere (although
  56 a few tests expect specific files in cwd and will let you know about
  57 this right away if they are not there). All of these are UNIX command
  58 line tools using only a console (tty). There are some common command
  59 line arguments described below. There is also a script in cspxutils
  60 named cspdvt which runs many of the programs in sequence, aborting on
  61 any error.
  62
  63 Note that if you're running the Security framework which
  64 you have built yourself, you'll need to set the DYLD_FRAMEWORK_PATH
  65 environment variable. Generally this will be set to the same
  66 value as the LOCAL_BUILD_DIR environment variable described above.
  67
  68 Some programs contain many command line options which are intended for
  69 use in poking around the CSP and setting up unusual or pesky conditions.
  70 Use the 'h' command line argument to get a list of available options for
  71 a given program. Most programs do something useful when run with no command
  72 line options (except for randTest, which is menu-driven, and attachLeak,
  73 which is interactive).
  74
  75 There is also a script called 'testall' which runs cspdvt twice - once for
  76 the raw CSP and once for the CSPDL ('D' argument), as well as the cltpdvt
  77 script in ../clxutils.
  78
  79   Common command line arguments
  80   -----------------------------
  81
  82   Note the absence of the standard UNIX '-' prefix for arguments. Each
  83   argument is specified separately (e.g., "symTest c D", NOT "symTest cD").
  84
  85   h  help, display usage
  86   q  quiet, minimal stdout activity; abort immediately on any error with
  87      nonzero exit status
  88   c  verbose, additional stdout info
  89   D  use AppleCSPDL, not AppleCSP (not all programs actually work this option).
  90
  91   The 'q', 'c', and 'D' arguments are passed down from the cspdvt script
  92   to individual tests.
  93
  94   Many programs also allow a loop count to be specified (l=loopCount).
  95   Specifying l=0 results in the test running forever. This option is
  96   not passed down from the cspdvt script. Use shell commands (while,
  97   etc.) for repeated runs of cspdvt.
  98
  99   Many programs offer a 'p' option which causes a pause in the operation
 100   every so often, waiting for a CR. This is used in conjunction with
 101   MallocDebug for analyzing memory leaks.
 102
 103   Note that when you do NOT specify 'q' (either to the cspdvt script or
 104   an individual program), any error encountered will result in a 'pause'
 105   condition, awaiting further interaction. To run cspdvt from another
 106   script, you need to specify 'q'. All programs and the cspdvt script
 107   return a nonzero exit status when they abort on error.
 108
 109   The cspdvt script also has a command line argument, 'l' (the letter l),
 110   which when specified, causes a longer, more extensive test run than
 111   normal. The normal test run currently takes about 1.5 minutes on a
 112   2 GHx Intel iMac. A 'long' test run takes about 10 minutes on the same machine.
 113   The cspdvt script passes through the 'D' argument to all programs which
 114   support it; thus 'cspdvt D' runs the entire suite on the CSPDL rather
 115   than the raw CSP.
 116
 117 Diagnosing a failure of cspdvt
 118 ------------------------------
 119
 120 A normal run of the cspdvt script - e.g. when run in the nightly build and
 121 in the buildbot environment - specifies the 'q' argument, which for all tests
 122 causes just a startup banner for each test to be printed; the banner shows the
 123 command line arguments passed to the tests. For example
 124
 125    # cspdvt q
 126    Starting ccSymCompat; args: q
 127    Starting ccSymCompat; args: o q
 128    Starting ccSymTest; args: q
 129    ...etc.
 130
 131 The first step in diagnosing a failure is generally to run the last test
 132 command seen in the list, without the 'q' option, in order to get more
 133 info about the failure.
 134
 135
 136 Description of tests
 137 -------------------
 138
 139 asymCompat : Test compatibility of CSP's RSA and DSA implementation with
 140                          the reference BSAFE library.
 141 asymTest   : test asymmetric encrypt/decrypt
 142 badattr    : Verifies proper behavior of CSP with respect to illegal attribute
 143              and usage bits in key headers
 144 badmac     : extensive MAC test which verifies detection of bad MACs
 145 badsig     : extensive signature test which verifies detection of bad
 146              signatures
 147 ccCtxSize  : Verify CommonCryptor context size constants
 148 ccHmacClone: Test cloning of CommonCrypto HMAC contexts
 149 ccHmacCompat: Verify compatibility of CommonCrypto HMAC with openssl.
 150 ccOneShot  : Test one-shot CommonDigest.
 151 ccSymCompat: Verify compatibility of CommonCryptor with reference implementations.
 152 ccSymTest  : extensive test of CommonCryptor modes and options.
 153 contextReuse : Verify proper operation of crypto context when they are reused
 154              for multiple operations.
 155 dhTest     : simple version of dhFullTest.
 156 dsaPartial : test partial DSA public key processing.
 157 hashClone  : Test CSSM_DigestDataClone()
 158 hashCompat : Test compatibility of CSP's digest implementations with
 159                          the reference BSAFE library.
 160 hashTest   : test digest functions
 161 keyDate    : test KeyHEader.{Start,End}Date handling.
 162 keyHash    : test the CSSM_APPLECSP_KEYDIGEST passthrough.
 163 keyHashAsym: extensive test of public/private key digest compare.
 164 keyStore   : Test persistence and retrieval of keys via CSPDL. Generally
 165                          requires user interaction via SecurityAgent.
 166 macCompat  : Test compatibility of CSP's MAC implementation with
 167                          the reference BSAFE library.
 168 macTest    : Basic MAC test.
 169 miniWrap   : brief and flexible key wrap/unwrap test
 170 pbeTest    : extensive test of DeriveKey operations
 171 rawRsaSig  : Test RAW RSA sign/verify by performing digest operations
 172              manually, doing sign/verify, and comparing result with
 173                          equivalent all-in-one op. Obsoleted by rawSig, which does
 174                          the same thing for all supported signature algorithms.
 175 rawSig     : Like rawRsaSig, for all algs.
 176 sigTest    : brief and flexible signature test
 177 ssl2Padding: test SSLv2 padding generation and detection
 178 symCompat  : verify compatibility of CSP symmetric encryption algorithms
 179              with reference implementations.
 180 symDelta   : verify that any change in any symmetric encryption parameter
 181              (one bit of the key, one bit of the IV, etc.) alters the
 182                          resulting ciphertext.
 183 symTest    : extensive test of symmetric encrypt/decrypt
 184 utilLib    : common routines used throughout the test suite.
 185 wrapTest   : extensive test of key wrap/unwrap.
 186
 187
 188 Tools, not run as part of cspdvt
 189 --------------------------------
 190
 191 aclTool    : Display and edit ACL entries.
 192 aesVect    : Generate test vectors, compare against NIST-provided references. Tests
 193                          CSP, standalone reference implementation, standalone Gladman.
 194 ascTool    : File-based tool to encrypt decrypt via ASC.
 195 asymPerform: Measure performance of asymmetric encryption algorithms.
 196 attachLeak : tool for analyzing leaks during CSSM_Init(), module load and
 197              attach.
 198 ccPerform  : Measure performance of CommonCrypto symmetric encryption.
 199 cryptTool  : Tool to demo simple CDSA crypto ops. Obsolete; replaced by similar
 200              program in SecurityTech/libCdsaCrypt module.
 201 dbTool     : Rudimentary tool to examine contents of a DB (a keychain).
 202 dbVerifyKey: Verify that specified DB has exactly one key of specified
 203                          algorithm, class, and key size - and no other keys.
 204 dhFullTest : file-based Diffie-Hellman exerciser/demo.
 205 genErrorStrings : Generate CSSSM error strings used in cssmErrToStr() to
 206                      isolate these tests from irregularitiues in similar functions
 207                          in Security.framework.
 208 genKeyPair : create a key pair, store in specified keychain.
 209 hashTime   : measure performance of Digest algorithms. A similar version,
 210                          hashTimeLibCrypt, measures the performance of the openssl
 211                          implementations. HashTimeSA measures the performance of
 212                          digest algs directly, with no framework overhead.
 213 keyStoreLeak: track down leaks involved in storing keys in DB.
 214 mdsdump    : Utility to dump various portions of the MDS databases.
 215 mdsLookup  : Example to demonstrate common usage of MDS lookups.
 216 perform    : measure performance of symmetric encrypt/decrypt.
 217 performRaw : Like perform, but not burdened by CssmClient.
 218 pubKeyTool : calculate public key hash of arbitrary keys and certs;
 219                          derive public key from a private key or a cert.
 220 randTest   : test the CSP's interface to the Yarrow pseudo random
 221              number generator. Interactive (menu-driven) and not
 222                          run as part of the cspdvt script. PRNGs are not easily
 223                          amenable to hands-off testing.
 224 rsaTool    : utility to perform sign, verify, encrypt, decrypt ops on
 225              raw files.
 226 sha2Time   : measure performance of SHA2 using CommonCrypto
 227 sha2Vectors, sha2VectorsCdsa - verify SHA2 implementation against NIST
 228                          test vectors using CommonCrypto and CDSA, respectively.
 229 sigPerform : measure performance of signature operations.
 230 symReference : write keys and ciphertext blobs, read them back
 231              and decrypt on (possibly) a different platfrom
 232 wrap       : utility to perform key derivation and key wrap/unwrap. Mainly
 233              used to verify interoperability between different versions
 234                          of CSP.
 235
 236
 237 There is also directory called utilLib which contains common code used by
 238 all of the tests in cspxutils. This library is mostly a set of C wrappers
 239 to make using the CSSM API a less painful experience than it otherwise is.