]> git.saurik.com Git - apple/security.git/blob - SecurityTests/clxutils/README
Security-57031.1.35.tar.gz
[apple/security.git] / SecurityTests / clxutils / README
1 clxutils Info
2 last update 10/11/06 dmitch
3
4 This directory contains a set of programs intended for use in testing the
5 following components of Security.framework on the OS X platform:
6
7 -- libsecurity_apple_x509_cl
8 -- libsecurity_apple_x509_tp
9 -- libsecurity_ssl
10 -- libsecurity_pkcs12
11 -- security_dotmactp
12 -- libsecurity_ocspd
13 -- import/export portion of libsecurity_keychain
14
15 The intended users are software engineers who are familiar with APIs for
16 the above modules. The programs are used for design verification testing
17 (DVT) and for interactive exercising of the above modules during bringup,
18 and for subsequent regression testing.
19
20 The intention is that there is within these tests are a way to exercise
21 every supported CL, TP, SSL, and import/export function. That's a tall
22 order; we give it our best shot.
23
24
25 Building
26 --------
27
28 Short version: to build everything in clxutils, cd to clxutils and
29 type 'make'. The resulting executables are placed in the location specified
30 in the LOCAL_BUILD_DIR environment variable (which you really must set to use
31 these tests effectively). If you haven't also built the cspxutils directory,
32 you need the libcsputils.a library built therein; see below.
33
34 Long version: The programs in clxutils build via standard Unix
35 Makefiles, not by Xcode. You generally want to set LOCAL_BUILD_DIR to
36 the same place Xcode places its built binaries ("Place Build Products in:"
37 in the Building pane in Xcode app's Preferences). This package builds OK
38 if you do not have $LOCAL_BUILD_DIR set, but the installed
39 Security.framework has to be up-to-date, and the cltpdvt script (see
40 below) will not run since executables are placed in their respective
41 source directories.
42
43 (Note: the number of times the LOCAL_BUILD_DIR environment variable is
44 mentioned here tends to lead an alert user of this code to just set that in
45 the user's .cshrc or equivalent and be done with it.)
46
47 The source for each program is contained in its own directory; you can
48 build a program on its own by typing 'make' in that directory. There is
49 also a top-level Makefile in clxutils so that you can go there and type
50 'make' and all of the included programs will build. The 'make clean'
51 target is supported at both levels.
52
53 Note that most of these program link against libcsputils.a, which
54 is built by SecurityTests/cspxutils/. Due to restrictions with
55 buildit and the Engineering build environment, the top-level Makefile
56 in clxutils does *not* ensure that libcsputils.a is built. You
57 have to do that. The easiest way to do that is, from the clxutils
58 directory:
59
60 # cd ../cspxutils/utilLib
61 # make
62
63 This source tree is generally up-to-date with top-of-tree Security.
64 No changes will be checked in to TOT clxutils until the corresponding
65 changes to Security, if any are needed, are also checked into TOT
66 Security.
67
68 Running
69 -------
70
71 Each executable can be run on its own, generally from anywhere (although
72 a few tests expect specific files in cwd and will let you know about
73 this right away if they are not there). All of these are UNIX command
74 line tools using only a console (tty). There are some common command
75 line arguments described below. There is also a script in clutils named
76 cltpdvt which runs many of the programs in sequence, aborting on any error.
77
78 The cltpdvt script has an 'l' option (the letter l) which causes a more
79 extensive set of thests to run. It also has an 'f' option which causes
80 a full SSL regression test to run. This includes building several keychains
81 in the current user's ~/Library/Keychains. In the absense of this, the
82 verifyPing script (in the sslViewer directory) is used to verify
83 client-side (only) SSL functionality.
84
85 Note that if you're running the Security framework which
86 you have built yourself, you'll need to set the DYLD_FRAMEWORK_PATH
87 environment variable. Generally this will be set to the same
88 value as the LOCAL_BUILD_DIR environment variable described above.
89
90 All programs display usage info if you just run them with the 'h' command
91 line argument. All programs which operate on certs expect the cert file
92 to be in raw BER-encoded form.
93
94 Diagnosing a failure of cltpdvt
95 -------------------------------
96
97 A normal run of the cltpdvt script - e.g. when run in the nightly build and
98 in the buildbot environment - specifies the 'q' argument, which for most tests
99 causes just a startup banner for each test to be printed; the banner shows the
100 command line arguments passed to the tests. For example
101
102 # cltpdvt q f
103 Starting cgConstruct; args: d=0 q
104 Starting cgConstruct; args: d=0 a=f q
105 Starting cgVerify; args: d n=2 q
106 ...etc.
107
108 The first step in diagnosing a failure is generally to run the last test
109 command seen in the list, without the 'q' option, in order to get more
110 info about the failure.
111
112 Note a few of the tests, notably the scripts in the sslScripts/ directory,
113 have a lot more console spew than just a basic startup banner. This spew
114 involves the command necessary to set up the SSL test environment, including
115 creating a number of keychain and populating them with certs and keys. In
116 case fo a failure in this case, look for the last instance of a "Starting..."
117 banner are restart from there.
118
119
120 Description of tests
121 -------------------
122
123 anchorTest : test cert decode/encode using system anchors.
124 caVerify : Tests proper verification of BasicConstraints extensions.
125 Written specifically for Radar 3022936.
126 certcrl : Industrial strength Power Tool tool to evaluate cert and CRL
127 collections. Allows for single command-line operation as well
128 as operation from a script file. Allows explicit testing of
129 every feature in the TP's CertGroupVerify function. See
130 README.doc in the certcrl directory for more info. There
131 are a number of scripts, located in certcrtl/testSubjects/, which
132 are used extensively by the top-level cltpdvt script to test
133 a wide variety of features.
134 certLabelTest : Test SecCertificateInferLabel()
135 certSerialEncodeTest : Verify proper encoding of unsigned integer as a DER_encoded signed integer.
136 Verifies Radar 4471281.
137 cgConstruct : Tests TP's CSSM_CertGroupConstruct().
138 cgVerify : Tests TP's CSSM_CertGroupVerify().
139 clAppUtils : common routines used through this test suite
140 extendAttrTest : test Extended keychain item attributes
141 extenTest : test all known extensions by generating random (but reasonable)
142 extensions, encoding them with the CL, decoding with the CL, and
143 displaying both pre-encode and post-encode versions.
144 extenTestTp : Same as extenTest, using CSSM_TP_SubmitCredRequest() to create
145 certs.
146 importExport : Exhaustive regression test of every line of code in the SecImportExport module
147 kcExport : export item(s) from a keychain, with extensive verification
148 kcImport : import item(s) into a keychain, with extensive verification
149 signerAndSubj : Build two certs, a signer and a subject which is signed by
150 the signer. Verify every which way, including operations
151 expected to result in verification failure.
152 signerAndSubjSsl : same as signerAndSubj, using CSSM_TP_SubmitCredRequest to
153 create the certs, with SSL-compatible extensions.
154 signerAndSubjTp : same as signerAndSubj, using CSSM_TP_SubmitCredRequest to
155 create the certs.
156 smimePolicy : Test CSSMOID_APPLE_TP_SMIME TP policy.
157 sslAlert : dual-threaded client/server test of SecureTransport's
158 alert message handling.
159 sslAuth : dual-threaded client/server test of SecureTransport's
160 client authentication logic.
161 sslCipher : Exhaustive dual-threaded client/server test of SecureTransport's
162 ciphersuite support
163 sslProt : dual-threaded client/server test of SecureTransport's
164 SSLProtocolVersion handling
165 sslScripts : Misc. scripts to generate certs, keychains, etc.
166 sslViewer : Power Tool to analyze the characteristics of an SSL server.
167 Many options to let you see the server's certs, the data
168 returned, negotiated SSL paramters, etc. This directory
169 also has a script, pingSslSites, which runs sslViewer on
170 a number of known (more-or-less) good SSL sites. See the
171 script for restrictions.
172 sslServer : Simple SecureTransport-based server.
173 sslSession : one-shot two-thread SSL client/server, limited options.
174 sslSubjName : Inelegant test of SecureTransport/TP verification of host
175 name verification
176 threadTest : fire up a bunch of threads, each of which executes a random
177 selection out of a large number of possible tests.
178
179
180 Tools, not run as part of cltpdvt
181 ---------------------------------
182 certChain : Given a cert, produce a complete ordered cert chain back to a root.
183 certInCrl : simple "see if cert is in CRL"
184 certsFromDb : Extracts all certs from a given DB (keychain) and writes
185 them to files.
186 certTime : measure performance of cert parse and build.
187 clTool : interactive "one step at a time" exerciser, mainly for
188 diagnosing memory leaks.
189 cmsTime : measure performance of CMS decode & verify
190 cmstool : manipluate CMS messages via libsecurity_smime SPI
191 crlTool : fetch and parse CRLs for given cert
192 dotMacArchive : test use of security_dotmactp to manipulate Identity archives on
193 the .mac server
194 dotMacRequest : exercise security_dotmactp at SecCertificateRequest level
195 dotMacTool : exercise basic security_dotmactp functions
196 extendAttrTool : view and manipulate extended keychain item attributes
197 extenGrab : Parses a cert file and writes out the DER-encoded
198 extension values for subsequent analysis and processing.
199 extractCertFields : A tool to extract fields from a cert file
200 and print them, on stdout, in C form. Used for embedding
201 root certs in the TP; obsolete.
202 findCert : Find all certs in keychain search list matching specified email address
203 idTool : SecIdentityRef exerciser
204 kcTime : measure performance of keychain operations
205 kcTool : Rudimentary tool to exercise exercise keychain support
206 keyFromCert : Tool to extract a public key from a cert file and write the
207 key blob to another file.
208 krbtool : basic PKINIT tool
209 makeCertPolicy : create a self signed cert with a Cert Policies extension
210 makeCrl : create a CRL revoking a given cert
211 newCmsTool : Power Tool to manipluate CMS messages via libsecurity_cms API
212 NISCC : tools for testing NISCC certs (see TLS_SSL/README.txt for more info).
213 ocspTool : simple OCSP request/response generator and parser
214 ocspdTool : basic ocspd tool
215 parseCert : display contents of a cert, to the best of the CL's ability to
216 parse it. Displays all known fields and extensions in parsed
217 form.
218 parseCrl : display contents of a CRL.
219 pemtool : encode and decode to/from PEM format.
220 p12 : PKCS12 tool. Obsolete for now since libsecurity_pkcs12 isn't viable outside of
221 Security.framework.
222 p12Reencode : test decode/reencode of p12 blobs. Currently not working.
223 pemtool : tool to convert files between PEM and DER format.
224 rootStoreTool : Trust Settings tool, including parser for trust settings records.
225 secTime : measure performance of various Sec layer functions.
226 secTrustTime : measure performance of SecTrust and TP cert verify
227 simpleUrlAccess : simple tool to exercise SecureTransport via URLAccess's
228 URLSimpleDownload(). URLSimpleDownload is deprecated, so this program is too.
229 sslBench : measure SSL handshake timing and throughput.
230 sslEAP : test EAP-FAST style PAC-based session resumption.
231 sslHandshakeTime : measure performance of SSLHandshake()
232 sslHandshakeTimeRB : measure performance of SSLHandshake() using RingBuffer I/O
233 sslScripts/ : futher SSL tests optionally run by cltpdvt.
234 makeLocalCert : builds the keychains and cert used by ssldvt
235 ssldvt : keychain-intensive test of SecureTransport
236 sslThroughput : Measure performance of SecureTransport - setup and sustained data throughput,
237 using RingBuffer I/O (no sockets).
238 sysIdTool : Tool for manipulating system identities.
239 trustApps : set list of trusted apps for specified executable
240 unBER : Fairly robust "Decode a BER-encoded file" program.
241 urlPageGrab : fetch a page of HTML, do a rudimentary parse looking for
242 "IMG SRC" tags, fetch each image in a separate thread. Uses
243 URLSimpleDownload, and is used for diagnosing failure from that
244 level on down to SecureTransport (inclusive).
245 userTrustTest : Rudimentary test of (deprecated) UserTrust mechanism.
246 vfyCacCert : verify a CAC certificate
247 vfyCert : verify one cert with another, or a root cert with itself.
248 vfyCertChain : verify a chain of certs with optional TP policies. Obsoleted
249 by certcrl.
250
251
252 Most programs use common code in clAppUtils. Many also use the common CSP code
253 in ../cspxutils/utilLib.