<jupyter_start><jupyter_text>Regular Expressions (RegEx) Las regex son cadenas de caracteres basadas en reglas sintácticas que permiten describir secuencias de caracteres. Así, forman parte de los lenguajes regulares, los cuales son un subgrupo de los lenguajes formales, de gran importancia para la tecnología de la información y, especialmente, para el desarrollo de software. Una expresión regular puede estar formada, o bien exclusivamente por caracteres normales (como abc), o bien por una combinación de caracteres normales y metacaracteres (como ab*c). Los metacaracteres describen ciertas construcciones o disposiciones de caracteres: por ejemplo, si un carácter debe estar en el inicio de la línea o si un carácter solo debe o puede aparecer exactamente una vez, más veces o menos. ![regex](https://miro.medium.com/max/1200/1*ZVlIZ1ZYC6rASz-dYPzhZQ.jpeg) **references**- https://docs.python.org/3/howto/regex.html- https://www.datacamp.com/community/tutorials/python-regular-expression-tutorial- https://www.dataquest.io/blog/regular-expressions-data-scientists/- https://www.kaggle.com/rtatman/fraudulent-email-corpus**may save your life**- https://regex101.com/  First things first For the standard case **import re** should be enough. For the later case **pip3 install regex** should install it.<jupyter_code>import re
import numpy as np
import pandas as pd<jupyter_output><empty_output><jupyter_text>Sintax Special Characters:- `.` Matches any character except a newline.- `^` Matches the start of the string.- `$` Matches the end of the string or just before the newline at the end of the string.- `*` Matches 0 or more repetitions of the preceding RE.- `+` Matches 1 or more repetitions of the preceding RE.- `?` Matches 0 or 1 repetitions of the preceding RE.- NOTA: re.M -> modo multilineahttps://docs.python.org/3/library/re.htmlre-syntax  Special Sequences:- **Literals** `a` - **Alternation** `a|b`- **Character sets** `[ab]`, `[^ab]`- **Wildcards** `.`- **Escape special characters** `\` (?,*,+,^,&)- **Ranges** `[a-d]`, `[1-9]`- **Quantifiers** `{2}`, `{2,}`, `{2,4}`, `?`, `*`, `+`- **Grouping** `()`- **Anchors** `^`, `$`- **Character classes** `\w`, `\d`, `\s`, `\n`, `\W`, `\D`, `\S`**\w** - Matches any alphanumeric character (digits and alphabets). Equivalent to `[a-zA-Z0-9_]`. By the way, underscore _ is also considered an alphanumeric character. **\d** - Matches any digit. Equivalent to `[0-9]` **\s** - Matches where a string contains any whitespace character. Equivalent to `[ \t\n\r\f\v]`**\W** - Matches any non-alphanumeric character. Equivalent to `[^a-zA-Z0-9_]`**\D** - Matches any non digit. Equivalent to `[^0-9]` **\S** - Matches where a string contains any non-whitespace character. Equivalent to `[^ \t\n\r\f\v]`  Methods  re.sub(pattern, repl, string, count=0)Replaces one or many matches with a string<jupyter_code>txt = "<NAME>, Borja and Clara are TA's??"
#re.sub
#Literals
re.sub(r"g",'G',txt)
#Ranges
re.sub(r'[A-Z]','',txt)
#Escape special character, quantifiers
re.sub('\?{2}','.',txt)<jupyter_output><empty_output><jupyter_text>re.search(pattern, string, flags=0)Scan through a string, looking for any location where this RE matches. If the search is succesful, `re.search()` returns a match object. Otherwise, it returns `None`.<jupyter_code>#re.search
txt = "The rain in Spain"
x = re.search(r"^The.*Spain$", txt) 
x
y = re.search("^Hola.*", txt) 
print(y)
#\b whole words only
x = re.search(r"\bS\w+", txt)
print(x)
print(x.span())
#returns a tuple containing the start-, and end positions of the match
print(x.start())
#contains the start position of the match
print(x.end())
#contains the end position of the match
print(x.string)
#print the string passed into the function (variable 'txt')
print(x.group())
#Print the part of the string where there was a match
print(re.search(r'r\w*', txt))
print(re.search(r'R\w*', txt))
print(re.search(r'^T\w*', txt))
print(re.search(r'^t\w*', txt))<jupyter_output><re.Match object; span=(4, 8), match='rain'>
None
<re.Match object; span=(0, 3), match='The'>
None<jupyter_text>re.match(pattern, string)Determine if the RE matches at the **beginning** of the string.<jupyter_code>#re.match
pattern = r"Cookie"

sequence = "I want a Cookie"
sequence2= "Cookie, I want you!"

if re.match(pattern, sequence):
    print("Match!")
else: 
    print("Not a match!")
txt = "The rain in Madrid"
#matches at the beginning of the string
print(re.match(r'r\w*', txt))
print(re.match(r'^r\w*', txt))
print(re.match(r'^T\w*', txt))
print(re.match(r'T\w*', txt))
email_address = 'Please contact us at: <EMAIL>'

match = re.search(r'([\.\w-]+)@([\w\.]+)', email_address)

if match:
    print(match.group()) # The whole matched text
    print(match.group(1)) # The username (group 1)
    print(match.group(2)) # The host (group 2)<jupyter_output><EMAIL>
support
thebridge.com<jupyter_text>re.fullmatch(pattern, string)<jupyter_code>class_names = ["Karina", "Marina", "Isabel", "Gina", "Xinru", "Sonia", "Marycruz"]

for name in class_names:
    if re.fullmatch("Isabel", name):
        print(f"{name} is desired name")
    else:
        print(f"{name} is not desired name")<jupyter_output>Karina is not desired name
Marina is not desired name
Isabel is desired name
Gina is not desired name
Xinru is not desired name
Sonia is not desired name
Marycruz is not desired name<jupyter_text>re.findall (pattern, string)Find all substrings where the RE matches, and returns them as a list.<jupyter_code>#re.findall
email_address = "Please contact us at: <EMAIL>, <EMAIL>"

#'addresses' is a list that stores all the possible match
addresses = re.findall(r'(([\w\.]+)@([\w\.-]+))', email_address)
addresses
addresses2 = re.findall(r'[\w\.]+@[\w\.-]+', email_address)
addresses2
lista = []
for a in addresses2:
    lista.append(re.search(r"[\w\.-]+", a).group())
lista
print(re.findall('[^aeiou\s]+',email_address))<jupyter_output>['Pl', 's', 'c', 'nt', 'ct', 's', 't:', 's', 'pp', 'rt.d', 't', '@d', 't', '-sc', 'nc', '.c', 'm,', 'xyz@th', 'br', 'dg', '.c', 'm']<jupyter_text>**¿Qué me van a devolver las siguientes dos líneas de código?**<jupyter_code>email_address = "Please contact us at: <EMAIL>, <EMAIL>"
print(re.findall('c\w*\s', email_address))
print(re.findall('^P\w*', email_address))<jupyter_output>['Please']<jupyter_text>------------------------------------------------------------------------------<jupyter_code>with open('info.txt', 'r') as file:
    client_info = file.read()
client_info[:150]
emails_clients=re.findall(r"[\w.]+@[\w.]+", client_info)
print(emails_clients[:5])
client_numbers = re.findall(r"((00)?\+?\d{2}-\d{3}-\d{3}-\d{3})", client_info)

#findall si añadimos un subgrupo va a buscar solo esa ocurrencia, entonces tendremos
#que meter todo el regex entre parentesis
print([tupla[0] for tupla in client_numbers][:5])<jupyter_output>['34-739-941-941', '34-278-870-242', '34-999-876-292', '34-345-887-949', '34-905-089-682']<jupyter_text>re.split(pattern, string, maxsplit=0)Returns a list where the string has been split at each match<jupyter_code>#re.split
sente = "Hello,\n Please, contact me the sooner.\n Thank you,\n Me"
print(sente)
reg = re.split(r"\n ", sente)
reg
" ".join(reg)<jupyter_output><empty_output><jupyter_text>re.compile(pattern)Compiles a RE into a regular expression object.<jupyter_code>name_check = re.compile(r"[^A-Za-z ]")
name = input("Please insert your name:")
while name_check.search(name):
    # it loops while if finds a match
    print("Please enter your name correctly!")
    name = input("Please insert your name:")
print("Finally mate, I thought you'd never do it")<jupyter_output>Please insert your name:Clara
Finally mate, I thought you'd never do it<jupyter_text>-----------------------------------------------------------------------------------------------------------  Some practice Now is your turn.  Simple validation of a usernamehttps://www.codewars.com/kata/56a3f08aa9a6cc9b75000023    Write a simple regex to validate a username. Allowed characters are:- lowercase letters,- numbers,- underscoreLength should be between 4 and 16 characters (both included).<jupyter_code># your solution
def validate_usr(username):
    #your code here
    pass
check_usernames = ["unusuario", "UNUSUARIO", "UN_usuario", "un_usuario76", "8934_38875aa"]
# Solucion Mauro's team
import re

def validate_usr(username):
    patron = r'[a-z0-9_]{4,16}'
    busqueda = re.match(patron, username)
    if busqueda:
        return True
    else:
        return False
check_usernamesbool = [validate_usr(e) for e in check_usernames]
check_usernamesbool
# Solucion Miguel's team
def validateusr(username):
    res = re.compile(r"[^a-z\d+_+]")
    while res.search(username) or (len(username)< 4 or len(username)>16):
        return False
    else:
        return True
check_usernamesbool = [validateusr(e) for e in check_usernames]
check_usernamesbool<jupyter_output><empty_output><jupyter_text>Regex validate PIN code https://www.codewars.com/kata/55f8a9c06c018a0d6e000132ATM machines allow 4 or 6 digit PIN codes and PIN codes cannot contain anything but exactly 4 digits or exactly 6 digits.If the function is passed a valid PIN string, return true, else return false.Examples:```python"1234"   -->  True"12345"  -->  False"a234"   -->  False```<jupyter_code># your solution
pins = ["8475","fklajkfd"]
def validate_pin(pin):
    #return true or false
    pass
# Solucion adri's team
def validate_pin(pin):
    return bool(re.fullmatch('(\d{4}|\d{6})', pin))
check_pinbool = [validate_pin(e) for e in pins]
check_pinbool
# Solucion marycruz's team
def validatepin(pin):
    #return true or false
    regex = re.compile(r'^\d{4}$|^\d{6}$')
    if regex.fullmatch(str(pin)):
        return True
    else:
        return False
check_pinbool = [validatepin(e) for e in pins]
check_pinbool<jupyter_output><empty_output><jupyter_text>-----------------------------------------------------------------------------------------------------------  The FBI challenge - https://www.fbi.gov/scams-and-safety/common-fraud-schemes/nigerian-letter-or-419-fraud- https://www.kaggle.com/rtatman/fraudulent-email-corpus It's your first day at the FBI office and your boss has just send you an `txt` file: `emails.txt`, she asked you to make some analysis but first of all, you need to get a dataframe like the following. You'll need some python knowledge and some regex for that goal.  ---------------------------------------------------------------------------------------------------------------------------  Since we are good people, here you have a proposed solution<jupyter_code>emails_info={}
fh = open("emails.txt", "r").read()
fh.count("From r")
contents = re.split(r"From r", fh)
contents[0]
contents.pop(0)<jupyter_output><empty_output><jupyter_text>- sender_email- sender_name- date_sent- time_sent- subject  Info Sender<jupyter_code>info_sender=[]
for i,e in enumerate(contents):
    try:
        info_sender.append(re.search("From:.*", e).group())
    except: 
        info_sender.append("not found")
len(info_sender)
#sender_email
emails_info['sender_email']=[]
for line in info_sender:
    res=re.findall(r'[\w\.]+@[\w\.-]+', line)
    if res:
        emails_info['sender_email'].append(res[0])
    else:
        emails_info['sender_email'].append(np.nan)
        
len(emails_info['sender_email'])
#sender name
emails_info['sender_name']=[]
for line in info_sender:
    res=re.findall(r':.*<', line)
    if res:
        emails_info['sender_name'].append(res[0][1:-1])
    else:
        emails_info['sender_name'].append(np.nan)
len(emails_info['sender_name'])<jupyter_output><empty_output><jupyter_text>Info Dates<jupyter_code>#DATES
dates=[]
for i,e in enumerate(contents):
    try:
        dates.append(re.search("Date:.*", e).group())
    except: 
        dates.append("not found")
len(dates)
#email date
emails_info['date_sent']=[]
for dat in dates:
    res=re.findall(r"\d+\s\w{3}\s\d+", dat)
    if res:
        emails_info['date_sent'].append(res[0])
    else:
        emails_info['date_sent'].append(np.nan)

len(emails_info['date_sent'])
emails_info['time_sent']=[]
for dat in dates:
    res=re.findall(r"\d{2}:\d{2}:\d{2}", dat)
    if res:
        emails_info['time_sent'].append(res[0])
    else:
        emails_info['time_sent'].append(np.nan)

len(emails_info['time_sent'])<jupyter_output><empty_output><jupyter_text>Subject<jupyter_code>subject=[]
for i,e in enumerate(contents):
    try:
        subject.append(re.search("Subject:.*", e).group())
    except: 
        subject.append("not found")
len(subject)
emails_info['subject']=[]
for sub in subject:
    res=re.findall(r":.*", sub)
    if res:
        emails_info['subject'].append(res[0][2:])
    else:
        emails_info['subject'].append(np.nan)

len(emails_info['subject'])<jupyter_output><empty_output><jupyter_text>Creating DataFrame<jupyter_code>df=pd.DataFrame(emails_info)
df.isnull().sum()
df.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1862335653518948, 'fraction_numerical': 0.016241299303944315, 'mean_word_length': 4.068992551940416, 'pattern_counts': {'":': 1, '<': 73, '<?xml version=': 0, '>': 76, 'https://': 11, 'lorem ipsum': 0, 'www.': 7, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '342757', 'license': 'MIT', 'n_tokens_mistral': 4656, 'n_tokens_neox': 4493, 'n_words': 1196, 'path': 'week5_EDA_scraping_eda_summary/day2_regex_SQL_II/theory/python/regex/regex_apr2021.ipynb', 'repo_name': 'paleomau/MGOL_BOOTCAMP', 'size': '13032'}