<jupyter_start><jupyter_text>Sorting and Pattern Matching  Learning Objectives* * ** Become comfortable with sorting (inline or copied) and the options you have* Explore fancy sorting with a built-in or custom function as a "key"* Learn ways to search and find patterns with regular expressions* See how to split up strings with regular expressions  Here are some people's names* How could you imagine wanting to sort them?<jupyter_code>people = ['<NAME>', '<NAME>', '<NAME>', '<NAME>']<jupyter_output><empty_output><jupyter_text>Here are some shoe prices in US dollars* What if you wanted to sort and convert to Euros at the same time?<jupyter_code>prices = [134.50, 49.99, 300.00, 12.50, 68.49]<jupyter_output><empty_output><jupyter_text>Sorting functions* Inline - no copying with `list.sort()`<jupyter_code># Create a list of 10 random integers
import random
a = random.sample(range(50), 10)
print('a:', a, '\n')

# Inline sorting
a.sort()
print('inline sorted a:', a, '\n')<jupyter_output><empty_output><jupyter_text>**To reverse:**  add `reverse=True` to `sort()` method * Sort and create a copy with `sorted`<jupyter_code># another list of 10 random integers
b = random.sample(range(50), 10)
print('b:', b)

# make a sorted copy of list
b_sorted = sorted(b)
print('sorted copy of b:', b_sorted)<jupyter_output><empty_output><jupyter_text>Will the following work with inline `sort` and/or copy version `sorted`?```pythonc = (3, 4, 2, 6, 5)```<jupyter_code># Try here<jupyter_output><empty_output><jupyter_text>Sorting efficiency* `sort()` is usually faster on lists than `sorted()` as it doesn't create an intermediate list* however, `sort()` method only works on lists whereas `sorted()` works on lists and enumerable collections* use `timeit` magic to do a speed test on your own (modify code above) EXERCISE 1:  Do the following and time each with `%timeit` magic* Create a "big" list of random integers (1000+) like below* Create a sorted copy* Sort the list inline* Sort the list inline in reverse order* Convert to a list of floating point numbers (like below)* Sort the float list inline* Sort the float list inline in reverse order---NB:  Must do a "deep copy" of original list before applying an inline fuction.  e.g. ```pythontest = a.copy()```The list and float conversion:```pythona = random.sample(range(10000), 1000) converted to float listb = [float(x) for x in a]```Which of the above operations are fastest/slowest and why?<jupyter_code># Try your solution here<jupyter_output><empty_output><jupyter_text>**Adding a *function* to the sorting process**You can* sort by built-in functions* sort by custom functions<jupyter_code>people = ['<NAME>', '<NAME>', '<NAME>', '<NAME>']<jupyter_output><empty_output><jupyter_text>Let's sort `people` from above by last name<jupyter_code># Our function (anonymous one here)
func = lambda s: s.split()[1]
    
func(people[0])
sorted(people, key = func) # does not modify the original<jupyter_output><empty_output><jupyter_text>```python Does this work for the inline "sort" method?___.sort(key = ___)```Copy and fill in blanks below.<jupyter_code># Try your solution here<jupyter_output><empty_output><jupyter_text>EXERCISE 2: Sort a list of random integers by their value on the sin curve```pythonimport mathimport random HINT: create an anonymous function  math.sin(x) is the conversion functiona = random.sample(range(25, 50), 10)```<jupyter_code># Try your solution here<jupyter_output><empty_output><jupyter_text>Behind the scenes of the `sorted()` function with a key* when using a key, `sorted()` creates an intermediate list or proxy  The `re` module* Most letters and characters match themselves, e.g. `super` will only match 'super' in a case-sensitive manner (we can make it case-insensitive though with an optional argument)* `re` module gives us the power to search and match specific patterns in strings* `re` gives us functions to find all occurences or iterate over all matches* `re` allows us to split strings and substitute based on patterns Regular expressions in Python* These are the characters (called metacharacters) which have special meaning in Python regex: `. ^ $ * + ? { } [ ] \ | ( )`* regex basics are not going to be covered here (please reference https://docs.python.org/3/howto/regex.html for a Python regex howto)* Here are examples of using regex with functions and methods in `re` module:<jupyter_code>import re

# match() finds first occurence of compiled pattern and returns a match object
p = re.compile('[A-z][a-z]')
m = p.match('AaAa')
print(m.group())

# search() includes the regex AND search string and returns a match object
s = re.search('[A-Z][a-z]', 'AaAa')
print(s.group())

# findall() includes the regex AND search string and returns a list of all matches
f = re.findall('[A-z][a-z]', 'AaAa')
print(f)

# print(f.index('Aa'))

# finditer() includes the regex AND search string and returns an iterable with all matches
i = re.finditer('[A-z][a-z]', 'AaAa')
for x in i:
    print((x.group(), x.span()))<jupyter_output><empty_output><jupyter_text>EXERCISE 3:  A small one surrounded by three body guards on either side---Using the following text and the `re` module, find all sets of letters that match this pattern: three upper case, one lower case, and three more uppercase letters consecutively (e.g.  ABCdEFG).  Then print all the "small ones" (middle lowercase letters).  What does it say?```pythontext = "kAewtloIKHbWJZNhHVGxXDiQmzjfcpYbzRPBoLPDSmUbCunkfxZWDZoUZMiGqhRRiUvGmYmvnJIHEmbTMUKLECKdCthezSYBpIEl"```Hint:  A *regex* could look like: `[A-Z][a-z][A-Z]` to match `AbC`<jupyter_code># Try your solution here<jupyter_output><empty_output><jupyter_text>Note about regex with backslashes* When trying to match a '\' using a string literal as the pattern, we must use '\\\\'* This is where raw strings come in handy making the search for '\' into the pattern '\\'* The output from matching, searching and finding is still a string literal* Take this example below where we search for sections in a latex document<jupyter_code># Here's a bit of latex
latex = """\section{Introduction}
Make it possible for all to write documents with \LaTeX{}!

\subsection{more introduction}
Go more in detail \ldots

\subsubsection{even more introduction}
come to the point \ldots

\paragraph{Paragraphs}
A paragraph is small but 

\subparagraph{Subparagraphs}
subparagraphs are smaller! 

\paragraph{Outline}
First we start with a little example of the article class, which is an 
important documentclass. But there would be other documentclasses like 
book \ref{book}, report \ref{report} and letter \ref{letter} which are 
described in Section \ref{documentclasses}. Finally, Section 
\ref{conclusions} gives the conclusions.



\section{Documentclasses} \label{documentclasses}"""

# Find section using a string literal
literal = '\\\\section'
found1 = re.findall(literal, latex)
print(found1)

# Find section using a raw string (preface with 'r')
raw = r'\\section'
found2 = re.findall(raw, latex)
print(found2)<jupyter_output><empty_output><jupyter_text>Spliting* `split()` method splits a string by applying a compiled pattern (like we do with `match()`)* It has an optional second argument (maxsplit) which will limit the number of splits starting at the beginning of the input string<jupyter_code>text = 'Happy 15th Birthday!  You are 15 going on 30.'

# Create pattern which recognizes digits
p = re.compile('\d+')

# Use split method on pattern with string as argument
words = p.split(text)
print(words)

# Now we can use the string.format method to reconstruct a new string
newwish = '{}60{}60{}30{}'.format(words[0], words[1], words[2], words[3])
print(newwish)
# Example of using maxplit with split()
text = 'grabxxfirstxthreexxxandxherexxxxisxxrest'
p = re.compile('x+')
p.split(text, maxsplit = 3)<jupyter_output><empty_output><jupyter_text>EXERCISE 4: Phonebook* using `split()` use the following text"""<NAME>: 310.222.3333 400 Holly AveSandra Bullock: 512.456.1789 200 52nd StTommy-<NAME>: 210.555.7777 321 Calahan Rd"""to create a list of dictionaries similar to this one:```python{'Bullock': {'address': '200 52nd St',  'first': 'Sandra',  'last': 'Bullock',  'phone': '512.456.1789'}, 'Jones': {'address': '321 Calahan Rd',  'first': 'Tommy-Lee',  'last': 'Jones',  'phone': '210.555.7777'}, 'Martin': {'address': '400 Holly Ave',  'first': 'Steve',  'last': 'Martin',  'phone': '310.222.3333'}}```e.g.```python Split on new linesentries = re.split('\n', text) A bit fancier split - parses a string by ':' or space up to 3 splitsre.split(":? ", entry, 3)```<jupyter_code># Try your solution here<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11819128021279056, 'fraction_numerical': 0.019659997687059096, 'mean_word_length': 5.566438876233865, 'pattern_counts': {'":': 1, '<': 71, '<?xml version=': 0, '>': 71, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '429839', 'license': 'MIT', 'n_tokens_mistral': 2872, 'n_tokens_neox': 2672, 'n_words': 1053, 'path': 'notebooks/04.SortAndPatternMatching.ipynb', 'repo_name': 'michhar/python4ds-notebooks', 'size': '8718'}