SecurityTests/clxutils/README

   1                                                   clxutils Info
   2                                    last update 10/11/06 dmitch
   3
   4 This directory contains a set of programs intended for use in testing the
   5 following components of Security.framework on the OS X platform:
   6
   7         -- libsecurity_apple_x509_cl
   8         -- libsecurity_apple_x509_tp
   9         -- libsecurity_ssl
  10         -- libsecurity_pkcs12
  11         -- security_dotmactp
  12         -- libsecurity_ocspd
  13         -- import/export portion of libsecurity_keychain
  14
  15 The intended users are software engineers who are familiar with APIs for
  16 the above modules. The programs are used for design verification testing
  17 (DVT) and for interactive exercising of the above modules during bringup,
  18 and for subsequent regression testing.
  19
  20 The intention is that there is within these tests are a way to exercise
  21 every supported CL, TP, SSL, and import/export function. That's a tall
  22 order; we give it our best shot.
  23
  24
  25 Building
  26 --------
  27
  28 Short version: to build everything in clxutils, cd to clxutils and
  29 type 'make'. The resulting executables are placed in the location specified
  30 in the LOCAL_BUILD_DIR environment variable (which you really must set to use
  31 these tests effectively). If you haven't also built the cspxutils directory,
  32 you need the libcsputils.a library built therein; see below.
  33
  34 Long version: The programs in clxutils build via standard Unix
  35 Makefiles, not by Xcode. You generally want to set LOCAL_BUILD_DIR to
  36 the same place Xcode places its built binaries ("Place Build Products in:"
  37 in the Building pane in Xcode app's Preferences). This package builds OK
  38 if you do not have $LOCAL_BUILD_DIR set, but the installed
  39 Security.framework has to be up-to-date, and the cltpdvt script (see
  40 below) will not run since executables are placed in their respective
  41 source directories.
  42
  43 (Note: the number of times the LOCAL_BUILD_DIR environment variable is
  44 mentioned here tends to lead an alert user of this code to just set that in
  45 the user's .cshrc or equivalent and be done with it.)
  46
  47 The source for each program is contained in its own directory; you can
  48 build a program on its own by typing 'make' in that directory. There is
  49 also a top-level Makefile in clxutils so that you can go there and type
  50 'make' and all of the included programs will build. The 'make clean'
  51 target is supported at both levels.
  52
  53 Note that most of these program link against libcsputils.a, which
  54 is built by SecurityTests/cspxutils/. Due to restrictions with
  55 buildit and the Engineering build environment, the top-level Makefile
  56 in clxutils does *not* ensure that libcsputils.a is built. You
  57 have to do that. The easiest way to do that is, from the clxutils
  58 directory:
  59
  60   # cd ../cspxutils/utilLib
  61   # make
  62
  63 This source tree is generally up-to-date with top-of-tree Security.
  64 No changes will be checked in to TOT clxutils until the corresponding
  65 changes to Security, if any are needed, are also checked into TOT
  66 Security.
  67
  68 Running
  69 -------
  70
  71 Each executable can be run on its own, generally from anywhere (although
  72 a few tests expect specific files in cwd and will let you know about
  73 this right away if they are not there). All of these are UNIX command
  74 line tools using only a console (tty). There are some common command
  75 line arguments described below. There is also a script in clutils named
  76 cltpdvt which runs many of the programs in sequence, aborting on any error.
  77
  78 The cltpdvt script has an 'l' option (the letter l) which causes a more
  79 extensive set of thests to run. It also has an 'f' option which causes
  80 a full SSL regression test to run. This includes building several keychains
  81 in  the current user's ~/Library/Keychains. In the absense of this, the
  82 verifyPing script (in the sslViewer directory) is used to verify
  83 client-side (only) SSL functionality.
  84
  85 Note that if you're running the Security framework which
  86 you have built yourself, you'll need to set the DYLD_FRAMEWORK_PATH
  87 environment variable. Generally this will be set to the same
  88 value as the LOCAL_BUILD_DIR environment variable described above.
  89
  90 All programs display usage info if you just run them with the 'h' command
  91 line argument. All programs which operate on certs expect the cert file
  92 to be in raw BER-encoded form.
  93
  94 Diagnosing a failure of cltpdvt
  95 -------------------------------
  96
  97 A normal run of the cltpdvt script - e.g. when run in the nightly build and
  98 in the buildbot environment - specifies the 'q' argument, which for most tests
  99 causes just a startup banner for each test to be printed; the banner shows the
 100 command line arguments passed to the tests. For example
 101
 102    # cltpdvt q f
 103    Starting cgConstruct; args: d=0 q
 104    Starting cgConstruct; args: d=0 a=f q
 105    Starting cgVerify; args: d n=2 q
 106    ...etc.
 107
 108 The first step in diagnosing a failure is generally to run the last test
 109 command seen in the list, without the 'q' option, in order to get more
 110 info about the failure.
 111
 112 Note a few of the tests, notably the scripts in the sslScripts/ directory,
 113 have a lot more console spew than just a basic startup banner. This spew
 114 involves the command necessary to set up the SSL test environment, including
 115 creating a number of keychain and populating them with certs and keys. In
 116 case fo a failure in this case, look for the last instance of a "Starting..."
 117 banner are restart from there.
 118
 119
 120 Description of tests
 121 -------------------
 122
 123 anchorTest                : test cert decode/encode using system anchors.
 124 caVerify          : Tests proper verification of BasicConstraints extensions.
 125                     Written specifically for Radar 3022936.
 126 certcrl                   : Industrial strength Power Tool tool to evaluate cert and CRL
 127                     collections. Allows for single command-line operation as well
 128                                         as operation from a script file. Allows explicit testing of
 129                                         every feature in the TP's CertGroupVerify function. See
 130                                         README.doc in the certcrl directory for more info. There
 131                                         are a number of scripts, located in certcrtl/testSubjects/, which
 132                                         are used extensively by the top-level cltpdvt script to test
 133                                         a wide variety of features.
 134 certLabelTest     : Test SecCertificateInferLabel()
 135 certSerialEncodeTest : Verify proper encoding of unsigned integer as a DER_encoded signed integer.
 136                                            Verifies Radar 4471281.
 137 cgConstruct           : Tests TP's CSSM_CertGroupConstruct().
 138 cgVerify              : Tests TP's CSSM_CertGroupVerify().
 139 clAppUtils                : common routines used through this test suite
 140 extendAttrTest    : test Extended keychain item attributes
 141 extenTest         : test all known extensions by generating random (but reasonable)
 142                     extensions, encoding them with the CL, decoding with the CL, and
 143                             displaying both pre-encode and post-encode versions.
 144 extenTestTp               : Same as extenTest, using CSSM_TP_SubmitCredRequest() to create
 145                                         certs.
 146 importExport      : Exhaustive regression test of every line of code in the SecImportExport module
 147 kcExport                  : export item(s) from a keychain, with extensive verification
 148 kcImport                  : import item(s) into a keychain, with extensive verification
 149 signerAndSubj     : Build two certs, a signer and a subject which is signed by
 150                     the signer. Verify every which way, including operations
 151                                 expected to result in verification failure.
 152 signerAndSubjSsl  : same as signerAndSubj, using CSSM_TP_SubmitCredRequest to
 153                     create the certs, with SSL-compatible extensions.
 154 signerAndSubjTp   : same as signerAndSubj, using CSSM_TP_SubmitCredRequest to
 155                     create the certs.
 156 smimePolicy               : Test CSSMOID_APPLE_TP_SMIME TP policy.
 157 sslAlert                  : dual-threaded client/server test of SecureTransport's
 158                                         alert message handling.
 159 sslAuth                   : dual-threaded client/server test of SecureTransport's
 160                                         client authentication logic.
 161 sslCipher                 : Exhaustive dual-threaded client/server test of SecureTransport's
 162                                         ciphersuite support
 163 sslProt                   : dual-threaded client/server test of SecureTransport's
 164                                         SSLProtocolVersion handling
 165 sslScripts                : Misc. scripts to generate certs, keychains, etc.
 166 sslViewer         : Power Tool to analyze the characteristics of an SSL server.
 167                     Many options to let you see the server's certs, the data
 168                                         returned, negotiated SSL paramters, etc. This directory
 169                                         also has a script, pingSslSites, which runs sslViewer on
 170                                         a number of known (more-or-less) good SSL sites. See the
 171                                         script for restrictions.
 172 sslServer         : Simple SecureTransport-based server.
 173 sslSession                : one-shot two-thread SSL client/server, limited options.
 174 sslSubjName               : Inelegant test of SecureTransport/TP verification of host
 175                                         name verification
 176 threadTest        : fire up a bunch of threads, each of which executes a random
 177                     selection out of a large number of possible tests.
 178
 179
 180 Tools, not run as part of cltpdvt
 181 ---------------------------------
 182 certChain                 : Given a cert, produce a complete ordered cert chain back to a root.
 183 certInCrl                 : simple "see if cert is in CRL"
 184 certsFromDb       : Extracts all certs from a given DB (keychain) and writes
 185                                         them to files.
 186 certTime                  : measure performance of cert parse and build.
 187 clTool            : interactive "one step at a time" exerciser, mainly for
 188                     diagnosing memory leaks.
 189 cmsTime           : measure performance of CMS decode & verify
 190 cmstool                   : manipluate CMS messages via libsecurity_smime SPI
 191 crlTool           : fetch and parse CRLs for given cert
 192 dotMacArchive     : test use of security_dotmactp to manipulate Identity archives on
 193                                         the .mac server
 194 dotMacRequest     : exercise security_dotmactp at SecCertificateRequest level
 195 dotMacTool                : exercise basic security_dotmactp functions
 196 extendAttrTool    : view and manipulate extended keychain item attributes
 197 extenGrab                 : Parses a cert file and writes out the DER-encoded
 198                                         extension values for subsequent analysis and processing.
 199 extractCertFields : A tool to extract fields from a cert file
 200                                 and print them, on stdout, in C form. Used for embedding
 201                                 root certs in the TP; obsolete.
 202 findCert                  : Find all certs in keychain search list matching specified email address
 203 idTool                    : SecIdentityRef exerciser
 204 kcTime                    : measure performance of keychain operations
 205 kcTool            : Rudimentary tool to exercise exercise keychain support
 206 keyFromCert       : Tool to extract a public key from a cert file and write the
 207                     key blob to another file.
 208 krbtool                   : basic PKINIT tool
 209 makeCertPolicy    : create a self signed cert with a Cert Policies extension
 210 makeCrl           : create a CRL revoking a given cert
 211 newCmsTool        : Power Tool to manipluate CMS messages via libsecurity_cms API
 212 NISCC             : tools for testing NISCC certs (see TLS_SSL/README.txt for more info).
 213 ocspTool                  : simple OCSP request/response generator and parser
 214 ocspdTool                 : basic ocspd tool
 215 parseCert         : display contents of a cert, to the best of the CL's ability to
 216                     parse it. Displays all known fields and extensions in parsed
 217                                 form.
 218 parseCrl                  : display contents of a CRL.
 219 pemtool                   : encode and decode to/from PEM format.
 220 p12                               : PKCS12 tool. Obsolete for now since libsecurity_pkcs12 isn't viable outside of
 221                                                 Security.framework.
 222 p12Reencode               : test decode/reencode of p12 blobs. Currently not working.
 223 pemtool           : tool to convert files between PEM and DER format.
 224 rootStoreTool     : Trust Settings tool, including parser for trust settings records.
 225 secTime                   : measure performance of various Sec layer functions.
 226 secTrustTime      : measure performance of SecTrust and TP cert verify
 227 simpleUrlAccess   : simple tool to exercise SecureTransport via URLAccess's
 228                     URLSimpleDownload(). URLSimpleDownload is deprecated, so this program is too.
 229 sslBench                  : measure SSL handshake timing and throughput.
 230 sslEAP            : test EAP-FAST style PAC-based session resumption.
 231 sslHandshakeTime  : measure performance of SSLHandshake()
 232 sslHandshakeTimeRB : measure performance of SSLHandshake() using RingBuffer I/O
 233 sslScripts/               : futher SSL tests optionally run by cltpdvt.
 234    makeLocalCert  : builds the keychains and cert used by ssldvt
 235    ssldvt                 : keychain-intensive test of SecureTransport
 236 sslThroughput     : Measure performance of SecureTransport - setup and sustained data throughput,
 237                     using RingBuffer I/O (no sockets).
 238 sysIdTool         : Tool for manipulating system identities.
 239 trustApps                 : set list of trusted apps for specified executable
 240 unBER             : Fairly robust "Decode a BER-encoded file" program.
 241 urlPageGrab       : fetch a page of HTML, do a rudimentary parse looking for
 242                     "IMG SRC" tags, fetch each image in a separate thread. Uses
 243                                         URLSimpleDownload, and is used for diagnosing failure from that
 244                                         level on down to SecureTransport (inclusive).
 245 userTrustTest     : Rudimentary test of (deprecated) UserTrust mechanism.
 246 vfyCacCert                : verify a CAC certificate
 247 vfyCert           : verify one cert with another, or a root cert with itself.
 248 vfyCertChain      : verify a chain of certs with optional TP policies. Obsoleted
 249                                         by certcrl.
 250
 251
 252 Most programs use common code in clAppUtils. Many also use the common CSP code
 253 in ../cspxutils/utilLib.