| 1 | '\" |
| 2 | '\" Copyright (c) 1998 Sun Microsystems, Inc. |
| 3 | '\" Copyright (c) 1999 Scriptics Corporation |
| 4 | '\" |
| 5 | '\" This software is copyrighted by the Regents of the University of |
| 6 | '\" California, Sun Microsystems, Inc., Scriptics Corporation, ActiveState |
| 7 | '\" Corporation and other parties. The following terms apply to all files |
| 8 | '\" associated with the software unless explicitly disclaimed in |
| 9 | '\" individual files. |
| 10 | '\" |
| 11 | '\" The authors hereby grant permission to use, copy, modify, distribute, |
| 12 | '\" and license this software and its documentation for any purpose, provided |
| 13 | '\" that existing copyright notices are retained in all copies and that this |
| 14 | '\" notice is included verbatim in any distributions. No written agreement, |
| 15 | '\" license, or royalty fee is required for any of the authorized uses. |
| 16 | '\" Modifications to this software may be copyrighted by their authors |
| 17 | '\" and need not follow the licensing terms described here, provided that |
| 18 | '\" the new terms are clearly indicated on the first page of each file where |
| 19 | '\" they apply. |
| 20 | '\" |
| 21 | '\" IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY |
| 22 | '\" FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES |
| 23 | '\" ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY |
| 24 | '\" DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE |
| 25 | '\" POSSIBILITY OF SUCH DAMAGE. |
| 26 | '\" |
| 27 | '\" THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, |
| 28 | '\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, |
| 29 | '\" FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE |
| 30 | '\" IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE |
| 31 | '\" NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR |
| 32 | '\" MODIFICATIONS. |
| 33 | '\" |
| 34 | '\" GOVERNMENT USE: If you are acquiring this software on behalf of the |
| 35 | '\" U.S. government, the Government shall have only "Restricted Rights" |
| 36 | '\" in the software and related documentation as defined in the Federal |
| 37 | '\" Acquisition Regulations (FARs) in Clause 52.227.19 (c) (2). If you |
| 38 | '\" are acquiring the software on behalf of the Department of Defense, the |
| 39 | '\" software shall be classified as "Commercial Computer Software" and the |
| 40 | '\" Government shall have only "Restricted Rights" as defined in Clause |
| 41 | '\" 252.227-7013 (c) (1) of DFARs. Notwithstanding the foregoing, the |
| 42 | '\" authors grant the U.S. Government and others acting in its behalf |
| 43 | '\" permission to use and distribute the software in accordance with the |
| 44 | '\" terms specified in this license. |
| 45 | '\" |
| 46 | '\" RCS: @(#) Id: re_syntax.n,v 1.3 1999/07/14 19:09:36 jpeek Exp |
| 47 | '\" |
| 48 | .so man.macros |
| 49 | .TH re_syntax n "8.1" Tcl "Tcl Built-In Commands" |
| 50 | .BS |
| 51 | .SH NAME |
| 52 | re_syntax \- Syntax of Tcl regular expressions. |
| 53 | .BE |
| 54 | |
| 55 | .SH DESCRIPTION |
| 56 | .PP |
| 57 | A \fIregular expression\fR describes strings of characters. |
| 58 | It's a pattern that matches certain strings and doesn't match others. |
| 59 | |
| 60 | .SH "DIFFERENT FLAVORS OF REs" |
| 61 | Regular expressions (``RE''s), as defined by POSIX, come in two |
| 62 | flavors: \fIextended\fR REs (``EREs'') and \fIbasic\fR REs (``BREs''). |
| 63 | EREs are roughly those of the traditional \fIegrep\fR, while BREs are |
| 64 | roughly those of the traditional \fIed\fR. This implementation adds |
| 65 | a third flavor, \fIadvanced\fR REs (``AREs''), basically EREs with |
| 66 | some significant extensions. |
| 67 | .PP |
| 68 | This manual page primarily describes AREs. BREs mostly exist for |
| 69 | backward compatibility in some old programs; they will be discussed at |
| 70 | the end. POSIX EREs are almost an exact subset of AREs. Features of |
| 71 | AREs that are not present in EREs will be indicated. |
| 72 | |
| 73 | .SH "REGULAR EXPRESSION SYNTAX" |
| 74 | .PP |
| 75 | Tcl regular expressions are implemented using the package written by |
| 76 | Henry Spencer, based on the 1003.2 spec and some (not quite all) of |
| 77 | the Perl5 extensions (thanks, Henry!). Much of the description of |
| 78 | regular expressions below is copied verbatim from his manual entry. |
| 79 | .PP |
| 80 | An ARE is one or more \fIbranches\fR, |
| 81 | separated by `\fB|\fR', |
| 82 | matching anything that matches any of the branches. |
| 83 | .PP |
| 84 | A branch is zero or more \fIconstraints\fR or \fIquantified atoms\fR, |
| 85 | concatenated. |
| 86 | It matches a match for the first, followed by a match for the second, etc; |
| 87 | an empty branch matches the empty string. |
| 88 | .PP |
| 89 | A quantified atom is an \fIatom\fR possibly followed |
| 90 | by a single \fIquantifier\fR. |
| 91 | Without a quantifier, it matches a match for the atom. |
| 92 | The quantifiers, |
| 93 | and what a so-quantified atom matches, are: |
| 94 | .RS 2 |
| 95 | .TP 6 |
| 96 | \fB*\fR |
| 97 | a sequence of 0 or more matches of the atom |
| 98 | .TP |
| 99 | \fB+\fR |
| 100 | a sequence of 1 or more matches of the atom |
| 101 | .TP |
| 102 | \fB?\fR |
| 103 | a sequence of 0 or 1 matches of the atom |
| 104 | .TP |
| 105 | \fB{\fIm\fB}\fR |
| 106 | a sequence of exactly \fIm\fR matches of the atom |
| 107 | .TP |
| 108 | \fB{\fIm\fB,}\fR |
| 109 | a sequence of \fIm\fR or more matches of the atom |
| 110 | .TP |
| 111 | \fB{\fIm\fB,\fIn\fB}\fR |
| 112 | a sequence of \fIm\fR through \fIn\fR (inclusive) matches of the atom; |
| 113 | \fIm\fR may not exceed \fIn\fR |
| 114 | .TP |
| 115 | \fB*? +? ?? {\fIm\fB}? {\fIm\fB,}? {\fIm\fB,\fIn\fB}?\fR |
| 116 | \fInon-greedy\fR quantifiers, |
| 117 | which match the same possibilities, |
| 118 | but prefer the smallest number rather than the largest number |
| 119 | of matches (see MATCHING) |
| 120 | .RE |
| 121 | .PP |
| 122 | The forms using |
| 123 | \fB{\fR and \fB}\fR |
| 124 | are known as \fIbound\fRs. |
| 125 | The numbers |
| 126 | \fIm\fR and \fIn\fR are unsigned decimal integers |
| 127 | with permissible values from 0 to 255 inclusive. |
| 128 | .PP |
| 129 | An atom is one of: |
| 130 | .RS 2 |
| 131 | .TP 6 |
| 132 | \fB(\fIre\fB)\fR |
| 133 | (where \fIre\fR is any regular expression) |
| 134 | matches a match for |
| 135 | \fIre\fR, with the match noted for possible reporting |
| 136 | .TP |
| 137 | \fB(?:\fIre\fB)\fR |
| 138 | as previous, |
| 139 | but does no reporting |
| 140 | (a ``non-capturing'' set of parentheses) |
| 141 | .TP |
| 142 | \fB()\fR |
| 143 | matches an empty string, |
| 144 | noted for possible reporting |
| 145 | .TP |
| 146 | \fB(?:)\fR |
| 147 | matches an empty string, |
| 148 | without reporting |
| 149 | .TP |
| 150 | \fB[\fIchars\fB]\fR |
| 151 | a \fIbracket expression\fR, |
| 152 | matching any one of the \fIchars\fR (see BRACKET EXPRESSIONS for more detail) |
| 153 | .TP |
| 154 | \fB.\fR |
| 155 | matches any single character |
| 156 | .TP |
| 157 | \fB\e\fIk\fR |
| 158 | (where \fIk\fR is a non-alphanumeric character) |
| 159 | matches that character taken as an ordinary character, |
| 160 | e.g. \e\e matches a backslash character |
| 161 | .TP |
| 162 | \fB\e\fIc\fR |
| 163 | where \fIc\fR is alphanumeric |
| 164 | (possibly followed by other characters), |
| 165 | an \fIescape\fR (AREs only), |
| 166 | see ESCAPES below |
| 167 | .TP |
| 168 | \fB{\fR |
| 169 | when followed by a character other than a digit, |
| 170 | matches the left-brace character `\fB{\fR'; |
| 171 | when followed by a digit, it is the beginning of a |
| 172 | \fIbound\fR (see above) |
| 173 | .TP |
| 174 | \fIx\fR |
| 175 | where \fIx\fR is |
| 176 | a single character with no other significance, matches that character. |
| 177 | .RE |
| 178 | .PP |
| 179 | A \fIconstraint\fR matches an empty string when specific conditions |
| 180 | are met. |
| 181 | A constraint may not be followed by a quantifier. |
| 182 | The simple constraints are as follows; some more constraints are |
| 183 | described later, under ESCAPES. |
| 184 | .RS 2 |
| 185 | .TP 8 |
| 186 | \fB^\fR |
| 187 | matches at the beginning of a line |
| 188 | .TP |
| 189 | \fB$\fR |
| 190 | matches at the end of a line |
| 191 | .TP |
| 192 | \fB(?=\fIre\fB)\fR |
| 193 | \fIpositive lookahead\fR (AREs only), matches at any point |
| 194 | where a substring matching \fIre\fR begins |
| 195 | .TP |
| 196 | \fB(?!\fIre\fB)\fR |
| 197 | \fInegative lookahead\fR (AREs only), matches at any point |
| 198 | where no substring matching \fIre\fR begins |
| 199 | .RE |
| 200 | .PP |
| 201 | The lookahead constraints may not contain back references (see later), |
| 202 | and all parentheses within them are considered non-capturing. |
| 203 | .PP |
| 204 | An RE may not end with `\fB\e\fR'. |
| 205 | |
| 206 | .SH "BRACKET EXPRESSIONS" |
| 207 | A \fIbracket expression\fR is a list of characters enclosed in `\fB[\|]\fR'. |
| 208 | It normally matches any single character from the list (but see below). |
| 209 | If the list begins with `\fB^\fR', |
| 210 | it matches any single character |
| 211 | (but see below) \fInot\fR from the rest of the list. |
| 212 | .PP |
| 213 | If two characters in the list are separated by `\fB\-\fR', |
| 214 | this is shorthand |
| 215 | for the full \fIrange\fR of characters between those two (inclusive) in the |
| 216 | collating sequence, |
| 217 | e.g. |
| 218 | \fB[0\-9]\fR |
| 219 | in ASCII matches any decimal digit. |
| 220 | Two ranges may not share an |
| 221 | endpoint, so e.g. |
| 222 | \fBa\-c\-e\fR |
| 223 | is illegal. |
| 224 | Ranges are very collating-sequence-dependent, |
| 225 | and portable programs should avoid relying on them. |
| 226 | .PP |
| 227 | To include a literal |
| 228 | \fB]\fR |
| 229 | or |
| 230 | \fB\-\fR |
| 231 | in the list, |
| 232 | the simplest method is to |
| 233 | enclose it in |
| 234 | \fB[.\fR and \fB.]\fR |
| 235 | to make it a collating element (see below). |
| 236 | Alternatively, |
| 237 | make it the first character |
| 238 | (following a possible `\fB^\fR'), |
| 239 | or (AREs only) precede it with `\fB\e\fR'. |
| 240 | Alternatively, for `\fB\-\fR', |
| 241 | make it the last character, |
| 242 | or the second endpoint of a range. |
| 243 | To use a literal |
| 244 | \fB\-\fR |
| 245 | as the first endpoint of a range, |
| 246 | make it a collating element |
| 247 | or (AREs only) precede it with `\fB\e\fR'. |
| 248 | With the exception of these, some combinations using |
| 249 | \fB[\fR |
| 250 | (see next |
| 251 | paragraphs), and escapes, |
| 252 | all other special characters lose their |
| 253 | special significance within a bracket expression. |
| 254 | .PP |
| 255 | Within a bracket expression, a collating element (a character, |
| 256 | a multi-character sequence that collates as if it were a single character, |
| 257 | or a collating-sequence name for either) |
| 258 | enclosed in |
| 259 | \fB[.\fR and \fB.]\fR |
| 260 | stands for the |
| 261 | sequence of characters of that collating element. |
| 262 | The sequence is a single element of the bracket expression's list. |
| 263 | A bracket expression in a locale that has |
| 264 | multi-character collating elements |
| 265 | can thus match more than one character. |
| 266 | .VS 8.2 |
| 267 | So (insidiously), a bracket expression that starts with \fB^\fR |
| 268 | can match multi-character collating elements even if none of them |
| 269 | appear in the bracket expression! |
| 270 | (\fINote:\fR Tcl currently has no multi-character collating elements. |
| 271 | This information is only for illustration.) |
| 272 | .PP |
| 273 | For example, assume the collating sequence includes a \fBch\fR |
| 274 | multi-character collating element. |
| 275 | Then the RE \fB[[.ch.]]*c\fR (zero or more \fBch\fP's followed by \fBc\fP) |
| 276 | matches the first five characters of `\fBchchcc\fR'. |
| 277 | Also, the RE \fB[^c]b\fR matches all of `\fBchb\fR' |
| 278 | (because \fB[^c]\fR matches the multi-character \fBch\fR). |
| 279 | .VE 8.2 |
| 280 | .PP |
| 281 | Within a bracket expression, a collating element enclosed in |
| 282 | \fB[=\fR |
| 283 | and |
| 284 | \fB=]\fR |
| 285 | is an equivalence class, standing for the sequences of characters |
| 286 | of all collating elements equivalent to that one, including itself. |
| 287 | (If there are no other equivalent collating elements, |
| 288 | the treatment is as if the enclosing delimiters were `\fB[.\fR'\& |
| 289 | and `\fB.]\fR'.) |
| 290 | For example, if |
| 291 | \fBo\fR |
| 292 | and |
| 293 | \fB\o'o^'\fR |
| 294 | are the members of an equivalence class, |
| 295 | then `\fB[[=o=]]\fR', `\fB[[=\o'o^'=]]\fR', |
| 296 | and `\fB[o\o'o^']\fR'\& |
| 297 | are all synonymous. |
| 298 | An equivalence class may not be an endpoint |
| 299 | of a range. |
| 300 | .VS 8.2 |
| 301 | (\fINote:\fR |
| 302 | Tcl currently implements only the Unicode locale. |
| 303 | It doesn't define any equivalence classes. |
| 304 | The examples above are just illustrations.) |
| 305 | .VE 8.2 |
| 306 | .PP |
| 307 | Within a bracket expression, the name of a \fIcharacter class\fR enclosed |
| 308 | in |
| 309 | \fB[:\fR |
| 310 | and |
| 311 | \fB:]\fR |
| 312 | stands for the list of all characters |
| 313 | (not all collating elements!) |
| 314 | belonging to that |
| 315 | class. |
| 316 | Standard character classes are: |
| 317 | .PP |
| 318 | .RS |
| 319 | .ne 5 |
| 320 | .nf |
| 321 | .ta 3c |
| 322 | \fBalpha\fR A letter. |
| 323 | \fBupper\fR An upper-case letter. |
| 324 | \fBlower\fR A lower-case letter. |
| 325 | \fBdigit\fR A decimal digit. |
| 326 | \fBxdigit\fR A hexadecimal digit. |
| 327 | \fBalnum\fR An alphanumeric (letter or digit). |
| 328 | \fBprint\fR An alphanumeric (same as alnum). |
| 329 | \fBblank\fR A space or tab character. |
| 330 | \fBspace\fR A character producing white space in displayed text. |
| 331 | \fBpunct\fR A punctuation character. |
| 332 | \fBgraph\fR A character with a visible representation. |
| 333 | \fBcntrl\fR A control character. |
| 334 | .fi |
| 335 | .RE |
| 336 | .PP |
| 337 | A locale may provide others. |
| 338 | .VS 8.2 |
| 339 | (Note that the current Tcl implementation has only one locale: |
| 340 | the Unicode locale.) |
| 341 | .VE 8.2 |
| 342 | A character class may not be used as an endpoint of a range. |
| 343 | .PP |
| 344 | There are two special cases of bracket expressions: |
| 345 | the bracket expressions |
| 346 | \fB[[:<:]]\fR |
| 347 | and |
| 348 | \fB[[:>:]]\fR |
| 349 | are constraints, matching empty strings at |
| 350 | the beginning and end of a word respectively. |
| 351 | '\" note, discussion of escapes below references this definition of word |
| 352 | A word is defined as a sequence of |
| 353 | word characters |
| 354 | that is neither preceded nor followed by |
| 355 | word characters. |
| 356 | A word character is an |
| 357 | \fIalnum\fR |
| 358 | character |
| 359 | or an underscore |
| 360 | (\fB_\fR). |
| 361 | These special bracket expressions are deprecated; |
| 362 | users of AREs should use constraint escapes instead (see below). |
| 363 | .SH ESCAPES |
| 364 | Escapes (AREs only), which begin with a |
| 365 | \fB\e\fR |
| 366 | followed by an alphanumeric character, |
| 367 | come in several varieties: |
| 368 | character entry, class shorthands, constraint escapes, and back references. |
| 369 | A |
| 370 | \fB\e\fR |
| 371 | followed by an alphanumeric character but not constituting |
| 372 | a valid escape is illegal in AREs. |
| 373 | In EREs, there are no escapes: |
| 374 | outside a bracket expression, |
| 375 | a |
| 376 | \fB\e\fR |
| 377 | followed by an alphanumeric character merely stands for that |
| 378 | character as an ordinary character, |
| 379 | and inside a bracket expression, |
| 380 | \fB\e\fR |
| 381 | is an ordinary character. |
| 382 | (The latter is the one actual incompatibility between EREs and AREs.) |
| 383 | .PP |
| 384 | Character-entry escapes (AREs only) exist to make it easier to specify |
| 385 | non-printing and otherwise inconvenient characters in REs: |
| 386 | .RS 2 |
| 387 | .TP 5 |
| 388 | \fB\ea\fR |
| 389 | alert (bell) character, as in C |
| 390 | .TP |
| 391 | \fB\eb\fR |
| 392 | backspace, as in C |
| 393 | .TP |
| 394 | \fB\eB\fR |
| 395 | synonym for |
| 396 | \fB\e\fR |
| 397 | to help reduce backslash doubling in some |
| 398 | applications where there are multiple levels of backslash processing |
| 399 | .TP |
| 400 | \fB\ec\fIX\fR |
| 401 | (where X is any character) the character whose |
| 402 | low-order 5 bits are the same as those of |
| 403 | \fIX\fR, |
| 404 | and whose other bits are all zero |
| 405 | .TP |
| 406 | \fB\ee\fR |
| 407 | the character whose collating-sequence name |
| 408 | is `\fBESC\fR', |
| 409 | or failing that, the character with octal value 033 |
| 410 | .TP |
| 411 | \fB\ef\fR |
| 412 | formfeed, as in C |
| 413 | .TP |
| 414 | \fB\en\fR |
| 415 | newline, as in C |
| 416 | .TP |
| 417 | \fB\er\fR |
| 418 | carriage return, as in C |
| 419 | .TP |
| 420 | \fB\et\fR |
| 421 | horizontal tab, as in C |
| 422 | .TP |
| 423 | \fB\eu\fIwxyz\fR |
| 424 | (where |
| 425 | \fIwxyz\fR |
| 426 | is exactly four hexadecimal digits) |
| 427 | the Unicode character |
| 428 | \fBU+\fIwxyz\fR |
| 429 | in the local byte ordering |
| 430 | .TP |
| 431 | \fB\eU\fIstuvwxyz\fR |
| 432 | (where |
| 433 | \fIstuvwxyz\fR |
| 434 | is exactly eight hexadecimal digits) |
| 435 | reserved for a somewhat-hypothetical Unicode extension to 32 bits |
| 436 | .TP |
| 437 | \fB\ev\fR |
| 438 | vertical tab, as in C |
| 439 | are all available. |
| 440 | .TP |
| 441 | \fB\ex\fIhhh\fR |
| 442 | (where |
| 443 | \fIhhh\fR |
| 444 | is any sequence of hexadecimal digits) |
| 445 | the character whose hexadecimal value is |
| 446 | \fB0x\fIhhh\fR |
| 447 | (a single character no matter how many hexadecimal digits are used). |
| 448 | .TP |
| 449 | \fB\e0\fR |
| 450 | the character whose value is |
| 451 | \fB0\fR |
| 452 | .TP |
| 453 | \fB\e\fIxy\fR |
| 454 | (where |
| 455 | \fIxy\fR |
| 456 | is exactly two octal digits, |
| 457 | and is not a |
| 458 | \fIback reference\fR (see below)) |
| 459 | the character whose octal value is |
| 460 | \fB0\fIxy\fR |
| 461 | .TP |
| 462 | \fB\e\fIxyz\fR |
| 463 | (where |
| 464 | \fIxyz\fR |
| 465 | is exactly three octal digits, |
| 466 | and is not a |
| 467 | back reference (see below)) |
| 468 | the character whose octal value is |
| 469 | \fB0\fIxyz\fR |
| 470 | .RE |
| 471 | .PP |
| 472 | Hexadecimal digits are `\fB0\fR'-`\fB9\fR', `\fBa\fR'-`\fBf\fR', |
| 473 | and `\fBA\fR'-`\fBF\fR'. |
| 474 | Octal digits are `\fB0\fR'-`\fB7\fR'. |
| 475 | .PP |
| 476 | The character-entry escapes are always taken as ordinary characters. |
| 477 | For example, |
| 478 | \fB\e135\fR |
| 479 | is |
| 480 | \fB]\fR |
| 481 | in ASCII, |
| 482 | but |
| 483 | \fB\e135\fR |
| 484 | does not terminate a bracket expression. |
| 485 | Beware, however, that some applications (e.g., C compilers) interpret |
| 486 | such sequences themselves before the regular-expression package |
| 487 | gets to see them, which may require doubling (quadrupling, etc.) the `\fB\e\fR'. |
| 488 | .PP |
| 489 | Class-shorthand escapes (AREs only) provide shorthands for certain commonly-used |
| 490 | character classes: |
| 491 | .RS 2 |
| 492 | .TP 10 |
| 493 | \fB\ed\fR |
| 494 | \fB[[:digit:]]\fR |
| 495 | .TP |
| 496 | \fB\es\fR |
| 497 | \fB[[:space:]]\fR |
| 498 | .TP |
| 499 | \fB\ew\fR |
| 500 | \fB[[:alnum:]_]\fR |
| 501 | (note underscore) |
| 502 | .TP |
| 503 | \fB\eD\fR |
| 504 | \fB[^[:digit:]]\fR |
| 505 | .TP |
| 506 | \fB\eS\fR |
| 507 | \fB[^[:space:]]\fR |
| 508 | .TP |
| 509 | \fB\eW\fR |
| 510 | \fB[^[:alnum:]_]\fR |
| 511 | (note underscore) |
| 512 | .RE |
| 513 | .PP |
| 514 | Within bracket expressions, `\fB\ed\fR', `\fB\es\fR', |
| 515 | and `\fB\ew\fR'\& |
| 516 | lose their outer brackets, |
| 517 | and `\fB\eD\fR', `\fB\eS\fR', |
| 518 | and `\fB\eW\fR'\& |
| 519 | are illegal. |
| 520 | .VS 8.2 |
| 521 | (So, for example, \fB[a-c\ed]\fR is equivalent to \fB[a-c[:digit:]]\fR. |
| 522 | Also, \fB[a-c\eD]\fR, which is equivalent to \fB[a-c^[:digit:]]\fR, is illegal.) |
| 523 | .VE 8.2 |
| 524 | .PP |
| 525 | A constraint escape (AREs only) is a constraint, |
| 526 | matching the empty string if specific conditions are met, |
| 527 | written as an escape: |
| 528 | .RS 2 |
| 529 | .TP 6 |
| 530 | \fB\eA\fR |
| 531 | matches only at the beginning of the string |
| 532 | (see MATCHING, below, for how this differs from `\fB^\fR') |
| 533 | .TP |
| 534 | \fB\em\fR |
| 535 | matches only at the beginning of a word |
| 536 | .TP |
| 537 | \fB\eM\fR |
| 538 | matches only at the end of a word |
| 539 | .TP |
| 540 | \fB\ey\fR |
| 541 | matches only at the beginning or end of a word |
| 542 | .TP |
| 543 | \fB\eY\fR |
| 544 | matches only at a point that is not the beginning or end of a word |
| 545 | .TP |
| 546 | \fB\eZ\fR |
| 547 | matches only at the end of the string |
| 548 | (see MATCHING, below, for how this differs from `\fB$\fR') |
| 549 | .TP |
| 550 | \fB\e\fIm\fR |
| 551 | (where |
| 552 | \fIm\fR |
| 553 | is a nonzero digit) a \fIback reference\fR, see below |
| 554 | .TP |
| 555 | \fB\e\fImnn\fR |
| 556 | (where |
| 557 | \fIm\fR |
| 558 | is a nonzero digit, and |
| 559 | \fInn\fR |
| 560 | is some more digits, |
| 561 | and the decimal value |
| 562 | \fImnn\fR |
| 563 | is not greater than the number of closing capturing parentheses seen so far) |
| 564 | a \fIback reference\fR, see below |
| 565 | .RE |
| 566 | .PP |
| 567 | A word is defined as in the specification of |
| 568 | \fB[[:<:]]\fR |
| 569 | and |
| 570 | \fB[[:>:]]\fR |
| 571 | above. |
| 572 | Constraint escapes are illegal within bracket expressions. |
| 573 | .PP |
| 574 | A back reference (AREs only) matches the same string matched by the parenthesized |
| 575 | subexpression specified by the number, |
| 576 | so that (e.g.) |
| 577 | \fB([bc])\e1\fR |
| 578 | matches |
| 579 | \fBbb\fR |
| 580 | or |
| 581 | \fBcc\fR |
| 582 | but not `\fBbc\fR'. |
| 583 | The subexpression must entirely precede the back reference in the RE. |
| 584 | Subexpressions are numbered in the order of their leading parentheses. |
| 585 | Non-capturing parentheses do not define subexpressions. |
| 586 | .PP |
| 587 | There is an inherent historical ambiguity between octal character-entry |
| 588 | escapes and back references, which is resolved by heuristics, |
| 589 | as hinted at above. |
| 590 | A leading zero always indicates an octal escape. |
| 591 | A single non-zero digit, not followed by another digit, |
| 592 | is always taken as a back reference. |
| 593 | A multi-digit sequence not starting with a zero is taken as a back |
| 594 | reference if it comes after a suitable subexpression |
| 595 | (i.e. the number is in the legal range for a back reference), |
| 596 | and otherwise is taken as octal. |
| 597 | .SH "METASYNTAX" |
| 598 | In addition to the main syntax described above, there are some special |
| 599 | forms and miscellaneous syntactic facilities available. |
| 600 | .PP |
| 601 | Normally the flavor of RE being used is specified by |
| 602 | application-dependent means. |
| 603 | However, this can be overridden by a \fIdirector\fR. |
| 604 | If an RE of any flavor begins with `\fB***:\fR', |
| 605 | the rest of the RE is an ARE. |
| 606 | If an RE of any flavor begins with `\fB***=\fR', |
| 607 | the rest of the RE is taken to be a literal string, |
| 608 | with all characters considered ordinary characters. |
| 609 | .PP |
| 610 | An ARE may begin with \fIembedded options\fR: |
| 611 | a sequence |
| 612 | \fB(?\fIxyz\fB)\fR |
| 613 | (where |
| 614 | \fIxyz\fR |
| 615 | is one or more alphabetic characters) |
| 616 | specifies options affecting the rest of the RE. |
| 617 | These supplement, and can override, |
| 618 | any options specified by the application. |
| 619 | The available option letters are: |
| 620 | .RS 2 |
| 621 | .TP 3 |
| 622 | \fBb\fR |
| 623 | rest of RE is a BRE |
| 624 | .TP 3 |
| 625 | \fBc\fR |
| 626 | case-sensitive matching (usual default) |
| 627 | .TP 3 |
| 628 | \fBe\fR |
| 629 | rest of RE is an ERE |
| 630 | .TP 3 |
| 631 | \fBi\fR |
| 632 | case-insensitive matching (see MATCHING, below) |
| 633 | .TP 3 |
| 634 | \fBm\fR |
| 635 | historical synonym for |
| 636 | \fBn\fR |
| 637 | .TP 3 |
| 638 | \fBn\fR |
| 639 | newline-sensitive matching (see MATCHING, below) |
| 640 | .TP 3 |
| 641 | \fBp\fR |
| 642 | partial newline-sensitive matching (see MATCHING, below) |
| 643 | .TP 3 |
| 644 | \fBq\fR |
| 645 | rest of RE is a literal (``quoted'') string, all ordinary characters |
| 646 | .TP 3 |
| 647 | \fBs\fR |
| 648 | non-newline-sensitive matching (usual default) |
| 649 | .TP 3 |
| 650 | \fBt\fR |
| 651 | tight syntax (usual default; see below) |
| 652 | .TP 3 |
| 653 | \fBw\fR |
| 654 | inverse partial newline-sensitive (``weird'') matching (see MATCHING, below) |
| 655 | .TP 3 |
| 656 | \fBx\fR |
| 657 | expanded syntax (see below) |
| 658 | .RE |
| 659 | .PP |
| 660 | Embedded options take effect at the |
| 661 | \fB)\fR |
| 662 | terminating the sequence. |
| 663 | They are available only at the start of an ARE, |
| 664 | and may not be used later within it. |
| 665 | .PP |
| 666 | In addition to the usual (\fItight\fR) RE syntax, in which all characters are |
| 667 | significant, there is an \fIexpanded\fR syntax, |
| 668 | available in all flavors of RE |
| 669 | with the \fB-expanded\fR switch, or in AREs with the embedded x option. |
| 670 | In the expanded syntax, |
| 671 | white-space characters are ignored |
| 672 | and all characters between a |
| 673 | \fB#\fR |
| 674 | and the following newline (or the end of the RE) are ignored, |
| 675 | permitting paragraphing and commenting a complex RE. |
| 676 | There are three exceptions to that basic rule: |
| 677 | .RS 2 |
| 678 | .PP |
| 679 | a white-space character or `\fB#\fR' preceded by `\fB\e\fR' is retained |
| 680 | .PP |
| 681 | white space or `\fB#\fR' within a bracket expression is retained |
| 682 | .PP |
| 683 | white space and comments are illegal within multi-character symbols |
| 684 | like the ARE `\fB(?:\fR' or the BRE `\fB\e(\fR' |
| 685 | .RE |
| 686 | .PP |
| 687 | Expanded-syntax white-space characters are blank, tab, newline, and |
| 688 | .VS 8.2 |
| 689 | any character that belongs to the \fIspace\fR character class. |
| 690 | .VE 8.2 |
| 691 | .PP |
| 692 | Finally, in an ARE, |
| 693 | outside bracket expressions, the sequence `\fB(?#\fIttt\fB)\fR' |
| 694 | (where |
| 695 | \fIttt\fR |
| 696 | is any text not containing a `\fB)\fR') |
| 697 | is a comment, |
| 698 | completely ignored. |
| 699 | Again, this is not allowed between the characters of |
| 700 | multi-character symbols like `\fB(?:\fR'. |
| 701 | Such comments are more a historical artifact than a useful facility, |
| 702 | and their use is deprecated; |
| 703 | use the expanded syntax instead. |
| 704 | .PP |
| 705 | \fINone\fR of these metasyntax extensions is available if the application |
| 706 | (or an initial |
| 707 | \fB***=\fR |
| 708 | director) |
| 709 | has specified that the user's input be treated as a literal string |
| 710 | rather than as an RE. |
| 711 | .SH MATCHING |
| 712 | In the event that an RE could match more than one substring of a given |
| 713 | string, |
| 714 | the RE matches the one starting earliest in the string. |
| 715 | If the RE could match more than one substring starting at that point, |
| 716 | its choice is determined by its \fIpreference\fR: |
| 717 | either the longest substring, or the shortest. |
| 718 | .PP |
| 719 | Most atoms, and all constraints, have no preference. |
| 720 | A parenthesized RE has the same preference (possibly none) as the RE. |
| 721 | A quantified atom with quantifier |
| 722 | \fB{\fIm\fB}\fR |
| 723 | or |
| 724 | \fB{\fIm\fB}?\fR |
| 725 | has the same preference (possibly none) as the atom itself. |
| 726 | A quantified atom with other normal quantifiers (including |
| 727 | \fB{\fIm\fB,\fIn\fB}\fR |
| 728 | with |
| 729 | \fIm\fR |
| 730 | equal to |
| 731 | \fIn\fR) |
| 732 | prefers longest match. |
| 733 | A quantified atom with other non-greedy quantifiers (including |
| 734 | \fB{\fIm\fB,\fIn\fB}?\fR |
| 735 | with |
| 736 | \fIm\fR |
| 737 | equal to |
| 738 | \fIn\fR) |
| 739 | prefers shortest match. |
| 740 | A branch has the same preference as the first quantified atom in it |
| 741 | which has a preference. |
| 742 | An RE consisting of two or more branches connected by the |
| 743 | \fB|\fR |
| 744 | operator prefers longest match. |
| 745 | .PP |
| 746 | Subject to the constraints imposed by the rules for matching the whole RE, |
| 747 | subexpressions also match the longest or shortest possible substrings, |
| 748 | based on their preferences, |
| 749 | with subexpressions starting earlier in the RE taking priority over |
| 750 | ones starting later. |
| 751 | Note that outer subexpressions thus take priority over |
| 752 | their component subexpressions. |
| 753 | .PP |
| 754 | Note that the quantifiers |
| 755 | \fB{1,1}\fR |
| 756 | and |
| 757 | \fB{1,1}?\fR |
| 758 | can be used to force longest and shortest preference, respectively, |
| 759 | on a subexpression or a whole RE. |
| 760 | .PP |
| 761 | Match lengths are measured in characters, not collating elements. |
| 762 | An empty string is considered longer than no match at all. |
| 763 | For example, |
| 764 | \fBbb*\fR |
| 765 | matches the three middle characters of `\fBabbbc\fR', |
| 766 | \fB(week|wee)(night|knights)\fR |
| 767 | matches all ten characters of `\fBweeknights\fR', |
| 768 | when |
| 769 | \fB(.*).*\fR |
| 770 | is matched against |
| 771 | \fBabc\fR |
| 772 | the parenthesized subexpression |
| 773 | matches all three characters, and |
| 774 | when |
| 775 | \fB(a*)*\fR |
| 776 | is matched against |
| 777 | \fBbc\fR |
| 778 | both the whole RE and the parenthesized |
| 779 | subexpression match an empty string. |
| 780 | .PP |
| 781 | If case-independent matching is specified, |
| 782 | the effect is much as if all case distinctions had vanished from the |
| 783 | alphabet. |
| 784 | When an alphabetic that exists in multiple cases appears as an |
| 785 | ordinary character outside a bracket expression, it is effectively |
| 786 | transformed into a bracket expression containing both cases, |
| 787 | so that |
| 788 | \fBx\fR |
| 789 | becomes `\fB[xX]\fR'. |
| 790 | When it appears inside a bracket expression, all case counterparts |
| 791 | of it are added to the bracket expression, so that |
| 792 | \fB[x]\fR |
| 793 | becomes |
| 794 | \fB[xX]\fR |
| 795 | and |
| 796 | \fB[^x]\fR |
| 797 | becomes `\fB[^xX]\fR'. |
| 798 | .PP |
| 799 | If newline-sensitive matching is specified, \fB.\fR |
| 800 | and bracket expressions using |
| 801 | \fB^\fR |
| 802 | will never match the newline character |
| 803 | (so that matches will never cross newlines unless the RE |
| 804 | explicitly arranges it) |
| 805 | and |
| 806 | \fB^\fR |
| 807 | and |
| 808 | \fB$\fR |
| 809 | will match the empty string after and before a newline |
| 810 | respectively, in addition to matching at beginning and end of string |
| 811 | respectively. |
| 812 | ARE |
| 813 | \fB\eA\fR |
| 814 | and |
| 815 | \fB\eZ\fR |
| 816 | continue to match beginning or end of string \fIonly\fR. |
| 817 | .PP |
| 818 | If partial newline-sensitive matching is specified, |
| 819 | this affects \fB.\fR |
| 820 | and bracket expressions |
| 821 | as with newline-sensitive matching, but not |
| 822 | \fB^\fR |
| 823 | and `\fB$\fR'. |
| 824 | .PP |
| 825 | If inverse partial newline-sensitive matching is specified, |
| 826 | this affects |
| 827 | \fB^\fR |
| 828 | and |
| 829 | \fB$\fR |
| 830 | as with |
| 831 | newline-sensitive matching, |
| 832 | but not \fB.\fR |
| 833 | and bracket expressions. |
| 834 | This isn't very useful but is provided for symmetry. |
| 835 | .SH "LIMITS AND COMPATIBILITY" |
| 836 | No particular limit is imposed on the length of REs. |
| 837 | Programs intended to be highly portable should not employ REs longer |
| 838 | than 256 bytes, |
| 839 | as a POSIX-compliant implementation can refuse to accept such REs. |
| 840 | .PP |
| 841 | The only feature of AREs that is actually incompatible with |
| 842 | POSIX EREs is that |
| 843 | \fB\e\fR |
| 844 | does not lose its special |
| 845 | significance inside bracket expressions. |
| 846 | All other ARE features use syntax which is illegal or has |
| 847 | undefined or unspecified effects in POSIX EREs; |
| 848 | the |
| 849 | \fB***\fR |
| 850 | syntax of directors likewise is outside the POSIX |
| 851 | syntax for both BREs and EREs. |
| 852 | .PP |
| 853 | Many of the ARE extensions are borrowed from Perl, but some have |
| 854 | been changed to clean them up, and a few Perl extensions are not present. |
| 855 | Incompatibilities of note include `\fB\eb\fR', `\fB\eB\fR', |
| 856 | the lack of special treatment for a trailing newline, |
| 857 | the addition of complemented bracket expressions to the things |
| 858 | affected by newline-sensitive matching, |
| 859 | the restrictions on parentheses and back references in lookahead constraints, |
| 860 | and the longest/shortest-match (rather than first-match) matching semantics. |
| 861 | .PP |
| 862 | The matching rules for REs containing both normal and non-greedy quantifiers |
| 863 | have changed since early beta-test versions of this package. |
| 864 | (The new rules are much simpler and cleaner, |
| 865 | but don't work as hard at guessing the user's real intentions.) |
| 866 | .PP |
| 867 | Henry Spencer's original 1986 \fIregexp\fR package, |
| 868 | still in widespread use (e.g., in pre-8.1 releases of Tcl), |
| 869 | implemented an early version of today's EREs. |
| 870 | There are four incompatibilities between \fIregexp\fR's near-EREs |
| 871 | (`RREs' for short) and AREs. |
| 872 | In roughly increasing order of significance: |
| 873 | .PP |
| 874 | .RS |
| 875 | In AREs, |
| 876 | \fB\e\fR |
| 877 | followed by an alphanumeric character is either an |
| 878 | escape or an error, |
| 879 | while in RREs, it was just another way of writing the |
| 880 | alphanumeric. |
| 881 | This should not be a problem because there was no reason to write |
| 882 | such a sequence in RREs. |
| 883 | .PP |
| 884 | \fB{\fR |
| 885 | followed by a digit in an ARE is the beginning of a bound, |
| 886 | while in RREs, |
| 887 | \fB{\fR |
| 888 | was always an ordinary character. |
| 889 | Such sequences should be rare, |
| 890 | and will often result in an error because following characters |
| 891 | will not look like a valid bound. |
| 892 | .PP |
| 893 | In AREs, |
| 894 | \fB\e\fR |
| 895 | remains a special character within `\fB[\|]\fR', |
| 896 | so a literal |
| 897 | \fB\e\fR |
| 898 | within |
| 899 | \fB[\|]\fR |
| 900 | must be written `\fB\e\e\fR'. |
| 901 | \fB\e\e\fR |
| 902 | also gives a literal |
| 903 | \fB\e\fR |
| 904 | within |
| 905 | \fB[\|]\fR |
| 906 | in RREs, |
| 907 | but only truly paranoid programmers routinely doubled the backslash. |
| 908 | .PP |
| 909 | AREs report the longest/shortest match for the RE, |
| 910 | rather than the first found in a specified search order. |
| 911 | This may affect some RREs which were written in the expectation that |
| 912 | the first match would be reported. |
| 913 | (The careful crafting of RREs to optimize the search order for fast |
| 914 | matching is obsolete (AREs examine all possible matches |
| 915 | in parallel, and their performance is largely insensitive to their |
| 916 | complexity) but cases where the search order was exploited to deliberately |
| 917 | find a match which was \fInot\fR the longest/shortest will need rewriting.) |
| 918 | .RE |
| 919 | |
| 920 | .SH "BASIC REGULAR EXPRESSIONS" |
| 921 | BREs differ from EREs in several respects. `\fB|\fR', `\fB+\fR', |
| 922 | and |
| 923 | \fB?\fR |
| 924 | are ordinary characters and there is no equivalent |
| 925 | for their functionality. |
| 926 | The delimiters for bounds are |
| 927 | \fB\e{\fR |
| 928 | and `\fB\e}\fR', |
| 929 | with |
| 930 | \fB{\fR |
| 931 | and |
| 932 | \fB}\fR |
| 933 | by themselves ordinary characters. |
| 934 | The parentheses for nested subexpressions are |
| 935 | \fB\e(\fR |
| 936 | and `\fB\e)\fR', |
| 937 | with |
| 938 | \fB(\fR |
| 939 | and |
| 940 | \fB)\fR |
| 941 | by themselves ordinary characters. |
| 942 | \fB^\fR |
| 943 | is an ordinary character except at the beginning of the |
| 944 | RE or the beginning of a parenthesized subexpression, |
| 945 | \fB$\fR |
| 946 | is an ordinary character except at the end of the |
| 947 | RE or the end of a parenthesized subexpression, |
| 948 | and |
| 949 | \fB*\fR |
| 950 | is an ordinary character if it appears at the beginning of the |
| 951 | RE or the beginning of a parenthesized subexpression |
| 952 | (after a possible leading `\fB^\fR'). |
| 953 | Finally, |
| 954 | single-digit back references are available, |
| 955 | and |
| 956 | \fB\e<\fR |
| 957 | and |
| 958 | \fB\e>\fR |
| 959 | are synonyms for |
| 960 | \fB[[:<:]]\fR |
| 961 | and |
| 962 | \fB[[:>:]]\fR |
| 963 | respectively; |
| 964 | no other escapes are available. |
| 965 | |
| 966 | .SH "SEE ALSO" |
| 967 | RegExp(3), regexp(n), regsub(n), lsearch(n), switch(n), text(n) |
| 968 | |
| 969 | .SH KEYWORDS |
| 970 | match, regular expression, string |