Regular expressions is the term used for a codified method of searching invented or defined by the American mathematician Stephen Kleene.
The syntax (language format) described is compliant with extended regular expressions (EREs) defined in IEEE POSIX 1003.2 (Section 2.8). EREs are now commonly supported by Apache, PHP4, Javascript 1.3+, MS Visual Studio, MS Frontpage, most visual editors, vi, emac, the GNU family of tools (including grep, awk and sed) as well as many others. Extended Regular Expressions (EREs) will support Basic Regular Expressions (BREs are essentially a subset of EREs). Most applications, utilities and laguages that implement RE's extend the capabilities defined. The appropriate documentation should always be consulted.
Some Definitions
- Liter
A literal is any character we use in a search or matching expression, for example, to find ind in windows the ind is a literal string - each character plays a part in the search, it is literally the string we want to find.
- Metacharacter
A metacharacter is one or more special characters that have a unique meaning
and are NOT used as literals in the search expression, for example, the character
^ (circumflex or caret) is a metacharacter.
- Escape sequence
An escape sequence is a way of indicating that we want to use one of our metacharacters as a literal. In a regular expression an escape sequence involves placing the metacharacter \ (backslash) in front of the metacharacter that we want to use as a literal, for example, if we want to find ^ind in w^indow then we use the search string \^ind and if we want to find file://file/ in the string c:\\file then we would need to use the search string \\\\file (each \ we want to search for (a literal) is preceded by an escape sequence \).
- Target string
This term describes the string that we will be searching, that is, the string in which we want to find our match or search pattern.
- Search expression
This term describes the expression that we will be using to search our target string, that is, the pattern we use to find what we want.
Brackets, Ranges and Negation
Bracket expressions introduce our first metacharacters, in this case the square brackets which allow us to define list of things to test for rather than the single characters we have been checking up until now. These lists can be grouped into what are known as Character Classes typically comprising well know groups such as all numbers etc.
[ ]
Match anything inside the square brackets for one character position once and only once, for example, [12] means match the target to either 1 or 2 while [0123456789] means match to any character in the range 0 to 9.
-
The - (dash) inside square brackets is the 'range separator' and allows us to define a range, in our example above of [0123456789] we could rewrite it as [0-9]
.
You can define more than one range inside a list e.g. [0-9A-C] means check for 0 to 9 and A to C (but not a to c).
Note: To test for - inside brackets (as a literal) it must come first or last, that is, [-0-9] will test for - and 0 to 9.
^
The ^ (circumflex or caret) inside square brackets negates the expression (we will see an alternate use for the circumflex/caret outside square brackets later), for example, [^Ff] means anything except upper or lower case F and [^a-z] means everything except lower case a to z.
Note:Spaces, or in this case the lack of them, between ranges are very important.
Positioning (or Anchors)
We can control where in our target strings the matches are valid. The following
is a list of metacharacters that affect the position of the search:
is a list of metacharacters that affect the position of the search:
^
The ^ (circumflex or caret) outside square brackets means look only at the beginning of the target string, for example, ^Win will not find Windows in string Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt) but ^Moz will find Mozilla.
$
The $ (dollar) means look only at the end of the target string, for example, fox$ will find a match in 'silver fox' since it appears at the end of the string but not in 'the fox jumped over the moon'.
.
The . (period) means any character(s) in this position, for example, ton. will find tons and tonneau but not wanton, wantons and ton because it has no following character.
NOTE:
Many systems and utilities, but not all, support special positioning macros, for example \< match at beginning of word, \> match at end of word, \b match at the begining OR end of word , \B except at the beginning or end of a word.
Many systems and utilities, but not all, support special positioning macros, for example \< match at beginning of word, \> match at end of word, \b match at the begining OR end of word , \B except at the beginning or end of a word.
List of the common values.
Character Class Abbreviations
\d
Match any character in the range 0 - 9 (equivalent of POSIX [:digit:])
\D
Match any character NOT in the range 0 - 9 (equivalent of POSIX [^[:digit:]])
\s
Match any whitespace characters (space, tab etc.). (equivalent of POSIX [:space:]
EXCEPT VT is not recognized)
EXCEPT VT is not recognized)
\S
Match any character NOT whitespace (space, tab). (equivalent of POSIX [^[:space:]])
\w
Match any character in the range 0 - 9, A - Z and a - z (equivalent of POSIX [:alnum:])
\W
Match any character NOT the range 0 - 9, A - Z and a - z (equivalent of POSIX [^[:alnum:]])
Positional Abbreviations
\b
Word boundary. Match any character(s) at the beginning (\bxx) and/or end (xx\b) of a word, thus \bton\b will find ton but not tons, but \bton will find tons.
\B
Not word boundary. Match any character(s) NOT at the beginning(\Bxx) and/or end (xx\B) of a word, thus \Bton\B will find wantons but not tons, but ton\B will find tons.
Iteration 'metacharacters'
The following is a set of iteration metacharacters (a.k.a. quantifiers) that can control the number of times a character or string is found in our searches.
?
The ? (question mark) matches the preceding character 0 or 1 times only, for example, colou?r will find both color and colour.
*
The * (asterisk or star) matches the preceding character 0 or more times, for example, tre* will find tree and tread and trough.
+
The + (plus) matches the previous character 1 or more times, for example, tre+ will find tree and tread but not trough.
{n}
Matches the preceding character n times exactly, for example, to find a local phone number we could use [0-9]{3}-[0-9]{4} which would find any number of the form 123-4567.
Note: The - (dash) in this case, because it is outside the square brackets, is a literal.Value is enclosed in braces (curly brackets).
{n,m}
Matches the preceding character at least n times but not more than m times, for example, 'ba{2,3}b' will find 'baab' and 'baaab' but NOT 'bab' or 'baaaab'. Values are enclosed in braces (curly brackets).
More 'metacharacters'
The following is a set of additional metacharacters that provide added power to our searches:
()
The ( (open parenthesis) and ) (close parenthesis) may be used to group (or bind) parts of our search expression together.
The (vertical bar or pipe) is called alternation in techspeak and means find the left hand OR right values, for example, gr(ae)y will find 'gray' or 'grey'.
<humblepie> In our examples, we blew this expression ^([L-Z]in), we incorrectly stated that this would negate the tests [L-Z], the '^' only performs this function inside square brackets, here it is outside the square brackets and is an anchor indicating 'start from first character'. Many thanks to Mirko Stojanovic for pointing it out and apologies to one and all.</humblepie>
