# Python Programming/Regular Expression

Python includes a module for working with regular expressions on
strings. For more information about writing regular expressions and
syntax not specific to Python, see the regular
expressions wikibook. Python\'s regular
expression syntax is similar to
Perl\'s

To start using regular expressions in your Python scripts, import the
\"re\" module:

``` python
import re
```

## Overview

Regular expression functions in Python at a glance:

``` python
import re
if re.search("l+","Hello"):        print(1) # Substring match suffices
if not re.match("ell.","Hello"):   print(2) # The beginning of the string has to match
if re.match(".el","Hello"):        print(3)
if re.match("he..o","Hello",re.I): print(4) # Case-insensitive match
print(re.sub("l+", "l", "Hello"))           # Prints "Helo"; replacement AKA substitution
print(re.sub(r"(.*)\1", r"\1", "HeyHey"))   # Prints "Hey"; backreference
print(re.sub("EY", "ey", "HEy", flags=re.I))# Prints "Hey"; case-insensitive sub
print(re.sub(r"(?i)EY", r"ey", "HEy"))      # Prints "Hey"; case-insensitive sub
for match in re.findall("l+.", "Hello Dolly"):
  print(match)                              # Prints "llo" and then "lly"
for match in re.findall("e(l+.)", "Hello Dolly"):
  print(match)                              # Prints "llo"; match picks group 1
for match in re.findall("(l+)(.)", "Hello Dolly"):
  print(match[0], match[1])                 # The groups end up as items in a tuple
match = re.match("(Hello|Hi) (Tom|Thom)","Hello Tom Bombadil")
if match:                                 # Equivalent to if match is not None
  print(match.group(0))                   # Prints the whole match disregarding groups
  print(match.group(1) + match.group(2))  # Prints "HelloTom"
```

## Matching and searching

One of the most common uses for regular expressions is extracting a part
of a string or testing for the existence of a pattern in a string.
Python offers several functions to do this.

The match and search functions do mostly the same thing, except that the
match function will only return a result if the pattern matches at the
beginning of the string being searched, while search will find a match
anywhere in the string.

``` python
>>> import re
>>> foo = re.compile(r'foo(.{,5})bar', re.I+re.S)
>>> st1 = 'Foo, Bar, Baz'
>>> st2 = '2. foo is bar'
>>> search1 = foo.search(st1)
>>> search2 = foo.search(st2)
>>> match1 = foo.match(st1)
>>> match2 = foo.match(st2)
```

In this example, match2 will be `None`, because the string `st2` does
not start with the given pattern. The other 3 results will be Match
objects (see below).

You can also match and search without compiling a regexp:

``` python
>>> search3 = re.search('oo.*ba', st1, re.I)
```

Here we use the search function of the re module, rather than of the
pattern object. For most cases, its best to compile the expression
first. Not all of the re module functions support the flags argument and
if the expression is used more than once, compiling first is more
efficient and leads to cleaner looking code.

The compiled pattern object functions also have parameters for starting
and ending the search, to search in a substring of the given string. In
the first example in this section, `match2` returns no result because
the pattern does not start at the beginning of the string, but if we do:

``` python
>>> match3 = foo.match(st2, 3)
```

it works, because we tell it to start searching at character number 3 in
the string.

What if we want to search for multiple instances of the pattern? Then we
have two options. We can use the start and end position parameters of
the search and match function in a loop, getting the position to start
at from the previous match object (see below) or we can use the findall
and finditer functions. The findall function returns a list of matching
strings, useful for simple searching. For anything slightly complex, the
finditer function should be used. This returns an iterator object, that
when used in a loop, yields Match objects. For example:

``` python
>>> str3 = 'foo, Bar Foo. BAR FoO: bar'
>>> foo.findall(str3)
[', ', '. ', ': ']
>>> for match in foo.finditer(str3):
...     match.group(1)
...
', '
'. '
': '
```

If you\'re going to be iterating over the results of the search, using
the finditer function is almost always a better choice.

### Match objects

Match objects are returned by the search and match functions, and
include information about the pattern match.

The group function returns a string corresponding to a capture group
(part of a regexp wrapped in `()`) of the expression, or if no group
number is given, the entire match. Using the `search1` variable we
defined above:

``` python
>>> search1.group()
'Foo, Bar'
>>> search1.group(1)
', '
```

Capture groups can also be given string names using a special syntax and
referred to by `matchobj.group('name')`. For simple expressions this is
unnecessary, but for more complex expressions it can be very useful.

You can also get the position of a match or a group in a string, using
the start and end functions:

``` python
>>> search1.start()
0
>>> search1.end()
8
>>> search1.start(1)
3
>>> search1.end(1)
5
```

This returns the start and end locations of the entire match, and the
start and end of the first (and in this case only) capture group,
respectively.

## Replacing

Another use for regular expressions is replacing text in a string. To do
this in Python, use the sub function.

sub takes up to 3 arguments: The text to replace with, the text to
replace in, and, optionally, the maximum number of substitutions to
make. Unlike the matching and searching functions, sub returns a string,
consisting of the given text with the substitution(s) made.

``` python
>>> import re
>>> mystring = 'This string has a q in it'
>>> pattern = re.compile(r'(a[n]? )(\w) ')
>>> newstring = pattern.sub(r"\1'\2' ", mystring)
>>> newstring
"This string has a 'q' in it"
```

This takes any single alphanumeric character (\\w in regular expression
syntax) preceded by \"a\" or \"an\" and wraps in in single quotes. The
`\1` and `\2` in the replacement string are backreferences to the 2
capture groups in the expression; these would be group(1) and group(2)
on a Match object from a search.

The subn function is similar to sub, except it returns a tuple,
consisting of the result string and the number of replacements made.
Using the string and expression from before:

``` python
>>> subresult = pattern.subn(r"\1'\2' ", mystring)
>>> subresult
("This string has a 'q' in it", 1)
```

Replacing without constructing and compiling a pattern object:

``` python
>>> result = re.sub(r"b.*d","z","abccde")
>>> result
'aze'
```

## Splitting

The split function splits a string based on a given regular expression:

``` python
>>> import re
>>> mystring = '1. First part 2. Second part 3. Third part'
>>> re.split(r'\d\.', mystring)
['', ' First part ', ' Second part ', ' Third part']
```

## Escaping

The escape function escapes all non-alphanumeric characters in a string.
This is useful if you need to take an unknown string that may contain
regexp metacharacters like `(` and `.` and create a regular expression
from it.

``` python
>>> re.escape(r'This text (and this) must be escaped with a "\" to use in a regexp.')
'This\\ text\\ \\(and\\ this\\)\\ must\\ be\\ escaped\\ with\\ a\\ \\"\\\\\\"\\ to\\ use\\ in\\ a\\ regexp\\.'
```

## Flags

The different flags use with regular expressions:

  Abbreviation   Full name         Description
  -------------- ----------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  `re.I`         `re.IGNORECASE`   Makes the regexp case-insensitive
  `re.L`         `re.LOCALE`       Makes the behavior of some special sequences (`\w, \W, \b, \B, \s, \S`) dependent on the current locale
  `re.M`         `re.MULTILINE`    Makes the `^` and `$` characters match at the beginning and end of each line, rather than just the beginning and end of the string
  `re.S`         `re.DOTALL`       Makes the `.` character match every character *including* newlines.
  `re.U`         `re.UNICODE`      Makes `\w, \W, \b, \B, \d, \D, \s, \S` dependent on Unicode character properties
  `re.X`         `re.VERBOSE`      Ignores whitespace except when in a character class or preceded by an non-escaped backslash, and ignores `#` (except when in a character class or preceded by an non-escaped backslash) and everything after it to the end of a line, so it can be used as a comment. This allows for cleaner-looking regexps.

## Pattern objects

If you\'re going to be using the same regexp more than once in a
program, or if you just want to keep the regexps separated somehow, you
should create a pattern object, and refer to it later when
searching/replacing.

To create a pattern object, use the compile function.

``` python
import re
foo = re.compile(r'foo(.{,5})bar', re.I+re.S)
```

The first argument is the pattern, which matches the string \"foo\",
followed by up to 5 of any character, then the string \"bar\", storing
the middle characters to a group, which will be discussed later. The
second, optional, argument is the flag or flags to modify the regexp\'s
behavior. The flags themselves are simply variables referring to an
integer used by the regular expression engine. In other languages, these
would be constants, but Python does not have constants. Some of the
regular expression functions do not support adding flags as a parameter
when defining the pattern directly in the function, if you need any of
the flags, it is best to use the compile function to create a pattern
object.

The `r` preceding the expression string indicates that it should be
treated as a raw string. This should normally be used when writing
regexps, so that backslashes are interpreted literally rather than
having to be escaped.

## External links

-   re --- Regular expression
    operations, docs.python.org
-   Python
    Programming/RegEx,
    wikiversity.org

fr:Programmation
Python/Regex
