<jupyter_start><jupyter_text>Regular Expressions  What are regular expressions? - patterns for matching text.  What can they be used for? - finding text. - finding and replacing text - `input` validation  Where are they used? - text editors and word processors. - search engines - text processing utilities e.g. `grep`, `sed` etc.  Raw strings in Python<jupyter_code>r'ab' == 'ab'
s = r'a\nb'
print(s)<jupyter_output>a\nb<jupyter_text>Using Regular Expressions in `Python`  Matching Ordinary Characters - characters like `'A'`, `'z'`, `'1'` match themselves   Example<jupyter_code>import re

text = 'this is a senTence9'

pattern = re.compile('is')
matches = pattern.finditer(text)

for match in matches:
    print(match, match.span())<jupyter_output><re.Match object; span=(2, 4), match='is'> (2, 4)
<re.Match object; span=(5, 7), match='is'> (5, 7)<jupyter_text>Using `re.compile`<jupyter_code>pattern<jupyter_output><empty_output><jupyter_text>Special Characters - stand for classes of ordinary character - affect how regular expressions around them are interpreted.  An overview  A real world example  Some useful functions  `re.match` - matches a pattern at the start of the string<jupyter_code>text = 'abcda'

pattern = re.compile('a')

re.match('a', text)
# pattern.match(text)
print(re.match('d', text))
re.match('ab', text)<jupyter_output><empty_output><jupyter_text>`re.search`<jupyter_code>text = 'abcda'

re.search('a', text)
re.search('d', text)
dir(match)
match = re.search('^a', text)

print(match.span(), match.re, match.string)<jupyter_output>(0, 1) re.compile('^a') abcda<jupyter_text>`re.match` v/s `re.search`<jupyter_code>text = 'aba'<jupyter_output><empty_output><jupyter_text>Difference in `MULTILINE` mode<jupyter_code>statement = 'A\nt'
print(statement)
re.search('^t', statement, re.M)
re.match('t', statement, re.M)<jupyter_output><empty_output><jupyter_text>`re.findall`<jupyter_code>text = 'abc5aba43a'

pattern = re.compile(r'\d')

print(pattern.findall(text))

pattern = re.compile(r'A')
match = pattern.findall(text)

print(len(match))<jupyter_output>0<jupyter_text>ProblemWrite a Python program to check that a string contains only a certain set of characters (in this case a-z, A-Z and 0-9).  Matching Character sets<jupyter_code>search = '''
12456
abcd
ABCD
**##
'''

pattern = re.compile(r'[0-9a-zA-Z*#]')

matches = pattern.finditer(search)

# print(type(matches))

for match in matches:
    print(match)<jupyter_output><class 'callable_iterator'>
<re.Match object; span=(1, 2), match='1'>
<re.Match object; span=(2, 3), match='2'>
<re.Match object; span=(3, 4), match='4'>
<re.Match object; span=(4, 5), match='5'>
<re.Match object; span=(5, 6), match='6'>
<re.Match object; span=(7, 8), match='a'>
<re.Match object; span=(8, 9), match='b'>
<re.Match object; span=(9, 10), match='c'>
<re.Match object; span=(10, 11), match='d'>
<re.Match object; span=(12, 13), match='A'>
<re.Match object; span=(13, 14), match='B'>
<re.Match object; span=(14, 15), match='C'>
<re.Match object; span=(15, 16), match='D'>
<re.Match object; span=(17, 18), match='*'>
<re.Match object; span=(18, 19), match='*'>
<re.Match object; span=(19, 20), match='#'>
<re.Match object; span=(20, 21), match='#'><jupyter_text>ProblemYou have been appointed as a library instructor in Godric's Hollow. Earlier, people used to do some sort of manual manipulations when they need to perform some task in the library. An example of such task could be trying to find the correct shelf where the books would be kept in an organized manner. Having had a bit of experience with Python, you want to chance your arm to handle these decisions programmatically. For each of the following situations you may assume a list of names of the books as input.Let us see how you can handle the following situations:- Your administrator want you to separate the name of books that matches a string that has an `'a' followed by zero or more 'b'.`- After some deliberation, the administrator seems to have change her mind. Now, she wants you to separate the books whose names have an `'a' followed by one or more 'b'`.- Now, happy with your previous skills, you are assigned another task, possible more possibly challenging. Can you separate the books whose names have `'a' followed by anything but ends in 'b'`?      Negating a character set<jupyter_code>text = 'abc234,ABC'
pattern = re.compile(r'[^a-zA-Z,0-2]')
matches = pattern.finditer(text)

for match in matches:
    print(match)<jupyter_output><re.Match object; span=(4, 5), match='3'>
<re.Match object; span=(5, 6), match='4'><jupyter_text>Using Quantifiers<jupyter_code>nums = '''
abcd
834.345.1254
892:345:3428
925-541-7625
548.326.4584
something else
4a3b4vd3d
'''

pattern = re.compile(r'\d\d\d.\d\d\d.\d\d\d\d')
matches = pattern.finditer(nums)

for match in matches:
    print(match)<jupyter_output><re.Match object; span=(6, 18), match='834.345.1254'>
<re.Match object; span=(19, 31), match='892:345:3428'>
<re.Match object; span=(32, 44), match='925-541-7625'>
<re.Match object; span=(45, 57), match='548.326.4584'><jupyter_text>Example<jupyter_code>names = '''
<NAME>
Dr. Smith
Dr- <NAME>
Sub. <NAME>
some gibberish
some more
'''

pattern = re.compile(r'(Er|Dr|Sub)[\.-] [\w ]+')
matches = pattern.finditer(names)

for match in matches:
    print(match)
websites = '''
http://hackadda.com
https://duck.com
https://www.python.org
'''

pattern = re.compile(r'https?://(www\.)?\w+\.(com|org)')
matches = pattern.finditer(websites)

for match in matches:
    print(match)<jupyter_output><re.Match object; span=(1, 20), match='http://hackadda.com'>
<re.Match object; span=(21, 37), match='https://duck.com'>
<re.Match object; span=(38, 60), match='https://www.python.org'><jupyter_text>`Groups`<jupyter_code>match = re.match(r"(\w+) (\w+), (\w+)", "Isaac Newton, physicist")
match.group(0)
match.group(1)
match.group(2)
match.groups()
match.group(0)<jupyter_output><empty_output><jupyter_text>ProblemWrite a Python program to convert a date of `yyyy-mm-dd` format to `dd-mm-yyyy` format.  Named Groups - `(?P)`<jupyter_code>match = re.match(r"(?P<first>\w+) (\w+), (\w+)", "Isaac Newton, physicist")
match.group('first')<jupyter_output><empty_output><jupyter_text>Group matches multiple times<jupyter_code>match = re.match(r"(..)+", "a1b2c3")
match.group(1)<jupyter_output><empty_output><jupyter_text>Replacing text with `re.sub`<jupyter_code>re.sub('[a-b]', '#', 'abcE')

list('abcd')
lst = list('abcd')

random.shuffle(lst)

lst
"".join(lst)
import random


def repl(m):
    inner_word = list(m.group(2))
    random.shuffle(inner_word)
    return m.group(1) + "".join(inner_word) + m.group(3)

text = "<NAME>, please report your absences promptly."
re.sub(r"(\w)(\w+)(\w)", repl, text)<jupyter_output><empty_output><jupyter_text>Flags |Flag | Meaning||-----|--------|ASCII, A| Makes several escapes like `\w`, `\b`, `\s` and `\d` match only on ASCII characters with the respective property.DOTALL, S| Make `.` match any character, including newlines.IGNORECASE, I| Do case-insensitive matches.LOCALE, L| Do a locale-aware match.MULTILINE, M| Multi-line matching, affecting `^` and `$`.VERBOSE, X (for ‘extended’)| Enable verbose REs, which can be organized more cleanly and understandably.<jupyter_code>a = re.compile(r"""\d +  # the integral part
                   \.    # the decimal point
                   \d*  # some fractional digits""", re.X)
b = re.compile(r"\d+\.\d*")
help('re.compile')<jupyter_output>Help on function compile in re:

re.compile = compile(pattern, flags=0)
    Compile a regular expression pattern, returning a Pattern object.<jupyter_text>Which to use: Regular Expressions or String method?<jupyter_code>s = 'a, b c'

s.replace(",", " ").split()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16612924029407972, 'fraction_numerical': 0.027860183154907778, 'mean_word_length': 5.334967320261438, 'pattern_counts': {'":': 0, '<': 98, '<?xml version=': 0, '>': 98, 'https://': 4, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '162289', 'license': 'MIT', 'n_tokens_mistral': 2917, 'n_tokens_neox': 2733, 'n_words': 765, 'path': 'chapter_13/regular_expressions.ipynb', 'repo_name': 'abhiabhi94/learn-python', 'size': '7794'}