<jupyter_start><jupyter_text>Python Programming: Data Structures and ClassesIn the previous lecture notebook we quickly coverd all of the importantcontrol structures available in Python, and how to define and usefunctions in Python.In this notebook we begin to look at some of the power of the high-levellanguage constructs Python provides by looking at some of the built-inbasic data types.  We will also cover the basics of defining and using classes and objects anddoing object-oriented programming in the Python language.  StringsStrings are not like numerical values.  In Pythong, strings or the `str` type are first-class data types of the language, unlike some othre languages.Strings are a fundamental type.Strings are a type of **sequence**, which means it is an order collectionof values.  Basically a string is an order collection of characters in Python.  A string is a sequenceYou can access individual characters in the string sequence using an indexingoperator (all sequences in Python provide indexing and slicing operations, as we will discuss).  Indexing is 0-based in Python, the first element is foundat index 0.<jupyter_code>fruit = 'banana'

# first character is at index 0
fruit[0]
# character at index 2 is the 3rd character, an n
letter = fruit[2]
letter<jupyter_output><empty_output><jupyter_text>You can use expressions that evaluate to an index to access characters ofa sequence like a string.<jupyter_code>length = 6 # banana has 6 characters
last_char = fruit[length - 1]
last_char
mid = length // 2
mid_char = fruit[mid]
mid_char<jupyter_output><empty_output><jupyter_text>'len'`len()` is a built-in function that returns the number of characters ina string (or in general, the number of items in any sequence container).<jupyter_code>len(fruit)<jupyter_output><empty_output><jupyter_text>To get the last letter of the string you might try this:<jupyter_code>length = len(fruit) # get the length programatically instead of hard coding it

try:
    last_char = fruit[length]
except IndexError:
    print('IndexError because valid indexes are from 0 to 5, but length is 6')
    print('   which is an invalid index.')<jupyter_output>IndexError because valid indexes are from 0 to 5, but length is 6
   which is an invalid index.<jupyter_text>So to get the last character you actually need the character at `length - 1`index.<jupyter_code>last_char = fruit[length - 1]
last_char<jupyter_output><empty_output><jupyter_text>Traversal with a `for` loopAnother way to iterate in Python is with `for` loops.  But unlike some languages,`for` loops are specifically designed to iterate over the items in a sequence.Since the items in a string are characters, if you iterate over a string you willget the individual characters of the string.<jupyter_code>for letter in fruit:
    print(letter)<jupyter_output>b
a
n
a
n
a<jupyter_text>You can compare this to manipulating an explicit index variable `index` to accessthe characters in a string.<jupyter_code>index = 0
while index < len(fruit):
    letter = fruit[index]
    print(index, letter)
    index = index + 1<jupyter_output>0 b
1 a
2 n
3 a
4 n
5 a<jupyter_text>It is considered bad style to use an index controlled loop like this to accessthe elementes of a sequence.  If you need to have both the index number of theitem in the sequence, and the item, use the built-in `enumerate()` function,which enumerates over the items of a sequence, returning the index and the itemas pairs, like this.<jupyter_code>for index, letter in enumerate(fruit):
    print(index, letter)<jupyter_output>0 b
1 a
2 n
3 a
4 n
5 a<jupyter_text>String slicesA segment of a string is called a slice.  **ALERT**: understanding slicing is very important for this class.  We use slicingof sequences in all kinds of contexts. You should learn all of the ways you can slice sequences, we will make heavyuse of slicing syntax in this class.  Many things we use in this class aresequences, like lists, numpy arrays, pandas dataframes, etc.<jupyter_code>s = '<NAME>'
# slice from the beginning to the 5th index (up to by not including index 5)
s[0:5]
# if you omit the beginning index of a slice, 0 is assumed
s[:5]
# slice from the middle of a sequence to the end
s[6:12]
# if  you omit the end index, all characters to the end are assumed
s[6:]
# arbitrary slice of characters from index 3 up to (but not including)
# index 8 in the middle of the sequence
s[3:8]
# given what we just demonstrated, what does the following mean?
s[:]
# you can provide a 3rd parameter for a slice, the step or skip
# get every even character of the string
s[::2]
# get every odd character starting at character 1
s[1::2]
# negative indexes are relative to the end of the sequence,
# get the last character
s[-1]
# get the last 2 characters
s[-2:]
# reverse the string, go from last to first character with a step of -1
s[::-1]<jupyter_output><empty_output><jupyter_text>Strings are immutableStrings are **immutable**.  This means you can't actually changethe characters of a string in place.   The other immutable data structurewe will use a lot is a **tuple**, which we will describe briefly later.What does it mean that strings are immutable.  Well you can't do something like this:<jupyter_code>greeting = 'Hello, world!'

try:
    greeting[0] = 'J'
except TypeError:
    print('TypeError is generated because it is illegal to try and change (mutate)')
    print('   the individual characters of a string sequence.')<jupyter_output>TypeError is generated because it is illegal to try and change (mutate)
   the individual characters of a string sequence.<jupyter_text>If you need to modify a string, the best you can do is create a newstring.<jupyter_code>new_greeting = 'J' + greeting[1:]
new_greeting<jupyter_output><empty_output><jupyter_text>Operations on stringsThere are operations that can be done on strings, like operations on numerictypes.  In another language, we might say that operations `+` and `*` are "overloaded" for the string type in the Python language.We just saw the operator `+` for strings.  Plus will concatenate strings.<jupyter_code># string concatenation
s = "Hello" + " " + "World!"
s<jupyter_output><empty_output><jupyter_text>The `*` operator on strings defines a string repetition.  The amount of repetitionhas to appear on the right hand side of the `*` operator.<jupyter_code># example of string repetition and concatenation together
x = 'Spam ' * 10 + "Spamity spam, spamity spam!"
x<jupyter_output><empty_output><jupyter_text>String methodsAlthough in many ways strings are fundamantal types in Python, it is morecorrect to think of strings as instances of objects.So an instance of a string has many member function that can be called to manipulate the string.  This is true of all of the rest of the high-level data types we will look at as well in the following sections.We can use the built-in method `dir()` to list all of the string methodsavailable for a string.  Documentation of [Python string methods](https://docs.python.org/3/library/stdtypes.htmlstring-methods)can be found at that link.<jupyter_code>s = 'A string'
dir(s)<jupyter_output><empty_output><jupyter_text>Just a few examples of string methods.<jupyter_code>s.upper()
s.find('s')
s.find('ing')
s.capitalize()
s.split()
s = '  \t   This string has lots of space, before and after it   \t\t\n'
s.strip()<jupyter_output><empty_output><jupyter_text>The `in` operatorThe word `in` is a boolean operator that takes a value and a sequenceand returns `True` if the value appears as an item in the sequence.For string types, `in` returns true if the value is a substring of the string.<jupyter_code>'nana' in 'banana'
'seed' in 'apple'
'seed' in 'birdseed and apple sauce'<jupyter_output><empty_output><jupyter_text>String comparisonBesides the `+` and `*` operations, all of the boolean operations aredefined for strings.  The boolean operators will compare strings by alphabetical ordering.<jupyter_code>'apple' < 'banana'
'grape' <= 'granite'
'apple' == 'orange'
'banana' == 'banana'
# but all upercase letters come before all lowercase letters, so you can be 
# surprised if you don't know this
'banana' < 'apple'
'Banana' < 'apple'<jupyter_output><empty_output><jupyter_text>Self-test Exercises  Lists  A list is a sequenceA list is also a sequence like a string.  However a list differs in twofundamental ways.1. A list is a sequence of arbitrary types/values, unlike a string which is a sequenceof characters.2. A list is **mutable**, which means unlike a string you can modify the elementsof a list in place.Lists look kind of like arrays if you are familiar with languages likeC or Java.  But they are really a much higher-level abstract data typethan a simple array.  It is best not to think of a list as a simple array.- Arrays are homogeneous, they only hold values of a single type.  Python lists are nonhomogeneous, they can hold values of many different types.- Arrays are fixed size, they cannot easily grow and shring.  Python listsare dynamically resizable, they can grow and shrink as needed.We create lists using the `[ ]` syntax.<jupyter_code># a list of integers
ivals = [10, 20, 30, 40]

# a list of strings
flavors = ['crunchy frog', 'ram bladder', 'lark vomit']

# but did we mention Python lists are nonhomogeneous
stuff = ['Gouda', 42, True, 4.2e-25, 2+3j, 'end of list']<jupyter_output><empty_output><jupyter_text>A value in a list can even be another list (e.g. nested lists), which canbe very useful for building complex data structures at times.<jupyter_code>l = ['spam', 2.0, 5, [10, 20, 30, 40]]
l<jupyter_output><empty_output><jupyter_text>Lists are sequences, so all of the things we talked about you could do with a sequence like a string, you can do with a list.<jupyter_code># index and slice list
l[-1]
l[1:3]
# iterate over a list
for index, element in enumerate(stuff):
    print(index, element)<jupyter_output>0 Gouda
1 42
2 True
3 4.2e-25
4 (2+3j)
5 end of list<jupyter_text>Lists are mutableBut lists are mutable, so we can change the items in a list.<jupyter_code>cheeses = ['Cheddar', 'Edam', 'Gouda']
cheeses[0] 
cheeses[0] = '<NAME>'
cheeses[-1] = 'Stilton'
cheeses<jupyter_output><empty_output><jupyter_text>Traversing a listYou should use the `for` loop construct to iterate over elementsof a Python list.  We showed one example above.  When you iterate over a nested list, you only get the top level elements, e.g.any nested lists will be extracted as a single element.  An exmaple should make this clearer<jupyter_code># l is  a nested list defined before
l
# if we iterate over l, the last element is the whole final list
for element in l:
    print(element)
# if you want to iterate over the nested elements, well...
for element in l:
    print(element)
    if type(element) == list:
        print("now displaying the individual elements of the nested list")
        for nested_element in element:
            print(nested_element)<jupyter_output>spam
2.0
5
[10, 20, 30, 40]
now displaying the individual elements of the nested list
10
20
30
40<jupyter_text>List operations`+` and `*` operators are also defined for list, and they do the same thingas for strings.  `+` concatenates lists.<jupyter_code>a = [1, 2, 3, 4]
b = [5, 6, 6, 8]

c = a + b
c<jupyter_output><empty_output><jupyter_text>And the `*` operator repeates a list<jupyter_code>c = a * 4
c
# create a bag of 10 red, green and blue balls for an experiment
balls = ['red', 'green', 'blue']
bag_of_balls = balls * 10
bag_of_balls<jupyter_output><empty_output><jupyter_text>List slicesAs we already mentioned, lists are sequences and all sequences supportslicing syntax in Python.<jupyter_code>t = ['a', 'b', 'c', 'd', 'e', 'f']
t[1:3]
t[:4]
t[3:]
t[::-1]
t[1:5:3]<jupyter_output><empty_output><jupyter_text>List methodsA Python list is an instance of an object.[Common sequence operations](https://docs.python.org/3/library/stdtypes.htmlcommon-sequence-operations)you can perform on a list.[Mutable sequence operations](https://docs.python.org/3/library/stdtypes.htmlmutable-sequence-types)you can perform on a list.A list is a mutable sequence, so any common sequence operation can be performedon lists, and any mutable sequence operations as well.We can get a list of the methods available to a list using `dir()` built-intmethod again.<jupyter_code>dir(t)<jupyter_output><empty_output><jupyter_text>Here are a few examples of using some of the methods you can perform on lists.<jupyter_code>t.reverse()
t
unsorted = ['grape', 'banana', 'apple', 'orange', 'zuchini', 'strawberry', 'kiwi']
unsorted.sort()
unsorted
unsorted.append('blueberry')
unsorted.append('rasberry')
unsorted
unsorted.sort()
unsorted
# using a list as a stack
stack = [] # initially empty

stack.append(5) # push
stack.append(3)
stack.append(7)
stack
top = stack.pop()
top
stack
stack.pop()
top = stack.pop()
top
stack
# using a list as a queue
queue = [] # initially empty
queue.append(5)
queue.append(3)
queue.append(7)

queue
front = queue.pop(0) # remove item 0, e.g. front of queue
front
queue
queue.pop(0)
front = queue.pop(0)
front
queue<jupyter_output><empty_output><jupyter_text>DictionariesDictionaries are a high-level abstract data type that you may or may notbe familiar with, but which are extremely useful to learn how to use.Dictionaries are also known as **maps**, **key-value pairs** or sometimesas a **hash**.  A dictionary is a mappingA dictionary is like a list but more general.  In a list, theinteger index is a key that maps from the index to the value at theindex for that list.  In a list the indices have to be integers.For the more general `dictionary`, a **key** can be any type, not just an integer.Each key is associated with a single value, just like each integer indexis associated with a single value in a list.However one difference, besides that the indexes can be arbitrary values, isthat since the keys don't necessarily have an inherent order, a dictionary isnot considered a sequence like a list.  In python `{` and `}` squiggly braces are used to identify and define dictionaries.<jupyter_code># dictionary of birthdays
birthday = {'Newton': 1642, 
            'Darwin': 1809, 
            'Curie': 1867, 
            'Einstein': 1879, 
            'Harter': 1978}
# look up some birthdays
birthday['Newton']
birthday['Einstein']<jupyter_output><empty_output><jupyter_text>If a key isn't in the dictionary you get an error, just like if you try andaccess a list index that is not in the list<jupyter_code>try:
    birthday['Hopper']
except KeyError:
    print('KeyError is generated here which means the key is not in the dictionary.')<jupyter_output>KeyError is generated here which means the key is not in the dictionary.<jupyter_text>The `len()` function works on dictionaries, so we can find out the number of key-value pairs in the dictionary using this.<jupyter_code>len(birthday)<jupyter_output><empty_output><jupyter_text>The `in` operator work on dictionaries too; it tell you whether something is a key of the dictionary or not.<jupyter_code>'Harter' in birthday
'Einstein' in birthday
'Hopper' in birthday<jupyter_output><empty_output><jupyter_text>Dictionaries are mutableDictionaries are mutable, like lists.  In fact, we can easily add newkey-value pairs.<jupyter_code>birthday['Hopper'] = 1912
birthday['Maxwell'] = 1875

birthday
len(birthday)
'Hopper' in birthday<jupyter_output><empty_output><jupyter_text>Dictionary methodsAs with lists and strings, dictionaries are actuall objects with a set of member methods that can be called on an instance of a dictionary.[Dictionary member methods](https://docs.python.org/3/library/stdtypes.htmlmapping-types-dict)<jupyter_code>dir(birthday)
birthday.keys()
birthday.values()
# iterate over the birthday by keys.
# Keys are not guaranteed to come out in any particular order from the dictionary/hash
for key in birthday.keys():
    print(key, birthday[key])
# actually, the default iteration of a dictionary is by the key, so
# if you want to iterate over the keys, you can use more simply
for key in birthday:
    print(key, birthday[key])
# we can iterate over the keys in an order, for example we could get the keys and
# sort them, then iterate over them
for key in sorted(birthday.keys()):
    print(key, birthday[key])
# items returns the key,value pairs together
for key, value in birthday.items():
    print(key, value)<jupyter_output>Newton 1642
Darwin 1809
Curie 1867
Einstein 1879
Harter 1978
Hopper 1912
Maxwell 1875<jupyter_text>Dictionaries to count valuesA common operation for data exploration is to get frequency counts of the typesand number of values of some data item.  Dictionaries are well suited for this task.For example, lets say you need to determine the count of the characters ina string.  We can use a dictionary and the `get()` method to do this easily.<jupyter_code>text = """
Just some random text we want to count character frequencies of.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor 
incididunt ut labore et dolore magna aliqua. Ipsum consequat nisl vel pretium 
lectus quam. Id leo in vitae turpis massa sed. Hendrerit gravida rutrum quisque 
non tellus orci ac auctor augue. Maecenas volutpat blandit aliquam etiam. Nibh 
cras pulvinar mattis nunc sed blandit libero volutpat sed. At erat pellentesque 
adipiscing commodo elit at imperdiet dui. Urna nec tincidunt praesent semper 
feugiat nibh sed. Sed pulvinar proin gravida hendrerit. Enim neque volutpat ac 
tincidunt vitae semper quis lectus nulla. Aenean pharetra magna ac placerat 
vestibulum lectus mauris. Hendrerit dolor magna eget est. Sapien et ligula 
ullamcorper malesuada proin libero nunc consequat.

Integer malesuada nunc vel risus commodo viverra. Vestibulum morbi blandit 
cursus risus at. Eleifend quam adipiscing vitae proin. Vulputate ut pharetra 
sit amet aliquam id. Neque ornare aenean euismod elementum. Et egestas quis 
ipsum suspendisse ultrices gravida dictum fusce ut. Enim sit amet venenatis 
urna. Morbi quis commodo odio aenean sed adipiscing. Justo nec ultrices dui 
sapien eget mi proin. Euismod in pellentesque massa placerat duis ultricies 
lacus sed turpis. Adipiscing bibendum est ultricies integer quis auctor. Vehicula 
ipsum a arcu cursus vitae congue mauris rhoncus. Facilisis volutpat est velit 
egestas dui id ornare arcu odio. Duis at consectetur lorem donec massa sapien 
faucibus et. Placerat in egestas erat imperdiet sed euismod nisi porta lorem.
"""
def character_frequency(text):
    frequencies = {} # keep frequency counts in a dictionary
    
    # iterate over the given text
    for char in text:
        count = frequencies.get(char, 0) # get current count or 0 if haven't seen yet
        count += 1 # increment count by 1
        frequencies[char] = count # update dictionary
        
    # return the dictionary result
    return frequencies
# get the character count/frequencies
freq = character_frequency(text)

# get the key/value pairs so we can sort by the value frequency counts
table = list(freq.items())

# sort the table.  We provide a custom sort function using a lambda function
# The lambda function essentially says for each item in table, sort by the
# second item at index 1, thus we sort by the frequency value 
table.sort(key=lambda key_value: key_value[1], reverse=True)

table<jupyter_output><empty_output><jupyter_text>TuplesA **tuple** is a sequence of values.  So a tuple is basically a list.However, you cannot modify a tuple once created, unlike a list.  Tuples are immutableA tuple is a sequence of values of any time.  You can index the valuesby an integer, and slice the tuple like a list.However, tuples are immutable.  Why do we need an immutable list/sequence typein Python?  Read on to find out.Syntatically you can create a tuple as a list of comma separatedvalues.  Also, it actually is not necessary in Python syntatically, butmany people indicate tuple lists by enclosing them in `(` and `)` (unlikea list where the square `[` and `]` are used and are actually necessarysyntatically).So we can create a tuple like:<jupyter_code>t1 = 'hello', 3, 3.14159, False, 42.42<jupyter_output><empty_output><jupyter_text>And although it is not necessary, you will commonly see the same tuple declaredlike<jupyter_code>t2 = ('hello', 3, 3.14159, False, 42.42)<jupyter_output><empty_output><jupyter_text>Most list operators (sequence operators) work on a tuple<jupyter_code>t2[0]
t2[1:3]
t1[-1]
for t in t2:
    print(t)<jupyter_output>hello
3
3.14159
False
42.42<jupyter_text>But if you try to modify one of the elements of a tuple you get an error<jupyter_code>try:
    t2[0] = 'goodbye'
except TypeError:
    print('TypeError generated becasue tuple object does not support item assignment.')<jupyter_output>TypeError generated becasue tuple object does not support item assignment.<jupyter_text>For a similar reason, methods like `append()` and `pop()` are not definedfor a tuple object as they would modify the tuple.Also as this may be useful for this class in places, the relational operatorswork with tuples (and other sequences as well).  Python starts by comparing thefirst element from each sequence.  If they are equal, it goes on to the next sequence.So our 2 tuples should evaluate as equal<jupyter_code>t1 == t2<jupyter_output><empty_output><jupyter_text>Or generally, the first value that differs in the tuple will determine the boolean relation.<jupyter_code>(1, 2, 3) < (1, 2, 4)
(1, 2, 3) < (1, 2, 2)<jupyter_output><empty_output><jupyter_text>Tuple assignmentTuples can assign individual elements to one another in anassignment.  This type of assignment between variables and itemsin a tuple is used extensively in Python and in the libraries we usefor this class.  For example:<jupyter_code>begin, end = 1, 100
print(begin)
print(end)
(name, age, salary, attributes) = ('Derek', 42, 10.00, {'height': 67, 'weight': 165})

print(name)
print(attributes['height'])
# kind of a trick, but we can use tuple assignment to perform an in place swap of 
# values
a, b = 42, 13

# do the swap
a, b = b, a

print(a)
print(b)<jupyter_output>13
42<jupyter_text>Tuples as return valuesProbably the most common use of this kind of tuple assignment is to handle thereturn values from a function.Strictly speaking, a function cn only return one value.  But if you want a function to return a list of values in Python, you can instead return atuple (or a list), and thus you are returning only 1 "thing", but the containercontains multiple values.  This is used extensively by library funciton to return multiple results back tothe caller of functions.  As a quick example, the built in `divmod()` function, that we have looked at before,actually returns a tuple result.  It returns the result of the division, followedby the remainder.  We can assign both return values into a single tuple.<jupyter_code># 7 / 3 = 2 with a remainder of 1
res = divmod(7, 3)
res<jupyter_output><empty_output><jupyter_text>Or instead, we can pick out the individual elements returned and assign them intomore useful variable names as a result of calling the function.<jupyter_code>quotient, remainder = divmod(7,3)
print('result of dividiong 7 by 3: ', quotient)
print('remainder from the division: ', remainder)<jupyter_output>result of dividiong 7 by 3:  2
remainder from the division:  1<jupyter_text>You can return multiple values from your own function by returning a tuplelist.  For example, here is a function that returns the min and max valuesfrom a list of values given to it.<jupyter_code>def min_max(l):
    """Given a list of values, return the min,max tuple of the minmum and 
    maximum values in the list.
    """
    return min(l), max(l)
# a list of integer values
l = [5, 3, 9, 2, 7, 6, 1, 8]

# call the list, get the tuple result
min_val, max_val = min_max(l)

print('minimum value in list:', min_val)
print('maximum value in list:', max_val)<jupyter_output>minimum value in list: 1
maximum value in list: 9<jupyter_text>Lists and tuples`zip()` is a built-in function that takes two or more sequences andinterleaves them. For example, `zip()` takes 2 tuples or lists of the same size, and this allowsyou to iterate over corresponding pairs of objects.<jupyter_code># a string is a sequence of characters, so it is "list" like
s = 'abcd'

# a tuple of 4 items
t = (1, 2, 3, 4)

for pair in zip(s, t):
    print(pair)
# once again because of tuple assignment, we could extract the individual
# items of the pair while iterating over them
for char, value in zip(s, t):
    print(char, value)<jupyter_output>a 1
b 2
c 3
d 4<jupyter_text>You can turn the rusult of calling zip into a true list, you can createsequences of corresponding elements.Also, `zip` will actually work with 3 or more sequences, as long as all sequencesare of the same length.<jupyter_code>names = ['alice', 'bob', 'carol']
ages = (42, 38, 29)
id = (1, 2, 3)

l = list(zip(id, names, ages))
l<jupyter_output><empty_output><jupyter_text>If you combine `zip`, `for` and tuple assignment, you get a useful idiomfor traversing two (or more) equences at the same time.  For example, `has_match`takes two sequences, `t1` and `t2`, and returns `True` if there is an indexi such that `t1[i] == t2[i]`.<jupyter_code>def has_match(t1, t2):
    """Determine if any corresponding element of sequence t1 matches t2.  If
    we find a matching element, we return a True result.  Otherwise the answer
    is false.
    """
    for x, y in zip(t1, t2):
        if x == y:
            return True
    return False
has_match((1, 2, 3), (4, 5, 3))
has_match( (1, 2, 3), (4, 5, 6))<jupyter_output><empty_output><jupyter_text>Finally, as we may have already mentioned, if you need to traverse the elementsof a sequence, like a list or tuple, and you also need the index of eachelement, you should use a `for` loop and the `enumerate()` function.  `enumerate()`basically zips a list of the integer indexes to the list, so  you canget each element and its index when iterating the list.<jupyter_code>for index, element in enumerate('abcdefg'):
    print(index, element)<jupyter_output>0 a
1 b
2 c
3 d
4 e
5 f
6 g<jupyter_text>Dictionaries and tuplesDictionaries have a method called `items()` that returns a sequence of tuples, where each tuple is a key-value pair.<jupyter_code>d = {'a': 0, 'b': 1, 'c': 2}
t = d.items()
t
for key, value in d.items():
    print(key, value)<jupyter_output>a 0
b 1
c 2<jupyter_text>Also it is common to use a tuple as a key for a complex dictionary.  One cannotuse lists because of their immutable nature.For example, we might need a telephone dictionary that maps `('last', 'first')` names to their telephone number.<jupyter_code>telephone_book = {
    ('Cleese', 'John'): '218-256-7918',
    ('Chapman', 'Graham'): '903-418-1938',
    ('Gilliam', 'Terry'): '817-763-8293',
    ('Palin', 'Michael'): '428-271-2939'
}

telephone_book['Gilliam', 'Terry']<jupyter_output><empty_output><jupyter_text>SetsSets are another built-in type that can be very useful inparticular circumstances.  A set is an implementation of themathematical notion of a set.  So each unique value only appears onetime within the set.We used a dictionary in a set like way, to count the occurrences ofletters in a text.  When a letter is encountered, we know if it is a letterin the text or not depending on if it is a key in the dictionary.To define a set do the following.<jupyter_code># an initially empty set
words = set() 
# using our lorem ipsum text from the sectionary on dictionaries, this time
# split by words, and create a set of the unique words
# By default, split() splits by whitespace, so spaces, tabs, newlines
for word in text.split():
    words.add(word)
# the set of unique words
words<jupyter_output><empty_output><jupyter_text>We can check set membership using the `in` operator, difference usingthe `difference()` method or the `-` operator, and set union using the `union()` or the `+` operator.<jupyter_code># set membership
'vel' in words
'velum' in words
# difference
s1 = set(['a', 'b', 'c', 'd'])
s2 = set(['b', 'c'])

s1 - s2
# the empty set
s2 - s1
s2.add('e')
s2 - s1
# union, whoops I was wrong, `+` is not defined for union, seems like a missed
# opportunity there?
#s1 + s2
s1.union(s2)<jupyter_output><empty_output><jupyter_text>List comprehensionsList comprehensions are a relatively newer addition to the Python language.On the one hand they can make you code more concise.  On the other hand they may make your code more concise, which may not be as readable?In any case, you will probably see list comprehensions used more and morefrequently in many places.  Any (simple) loop can be turned into a list comprehension, and any list comprehension can be turned back into a more traditional for loop.<jupyter_code># function to capitalize all letters of a string s, using a traditiona loop
def capitalize_all_loop(s):
    """Capitalize all letters in the string s and return the (new)
    resulting string.
    """
    res = []
    for c in s:
        res.append(c.capitalize())
    return res
capitalize_all_loop("This is a string!  It has some upper, but mostly lower.")<jupyter_output><empty_output><jupyter_text>Notice that the result is no a list instead of a string, which you may nothave been expecting.  How would you convert it back to a string?To do the same thing using a list comprehension, we do the following.Notice that `[` and `]` are defining a new list that we return, but inside ofthe list comprehension, instead of a comma separated list of elements, wehave a function/code, and a for statement.  The for statement selects each c ins, then `capitalize()` is called on each c, and the result of each of these isappended to the list that is created and returned.<jupyter_code>def capitalize_all_list(s):
    return [c.capitalize() for c in s]
capitalize_all_list("This is a string!  It has some upper, but mostly lower.")<jupyter_output><empty_output><jupyter_text>List comprehensions can also be used for filtering.  For example this function selects only the elements of `s` that are upper case, andreturns a new list:<jupyter_code>def only_upper_loop(s):
    """Select only the upper case letters of string s, and put them into a new
    list that we return to the caller.
    """
    res = []
    for c in s:
        if c.isupper():
            res.append(c)
    return res
only_upper_loop("This is a string!  It has some uPPER, but mostly lower.")<jupyter_output><empty_output><jupyter_text>The equivalant filter as a list comprehension<jupyter_code>def only_upper_list(s):
    return [c for c in s if c.isupper()]
only_upper_list("This is a string!  It has some uPPER, but mostly lower.")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07781618942012915, 'fraction_numerical': 0.014415237101298023, 'mean_word_length': 4.745362563237774, 'pattern_counts': {'":': 0, '<': 217, '<?xml version=': 0, '>': 210, 'https://': 4, 'lorem ipsum': 1, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '56', 'dir': 'jupyter-structured-clean-dedup', 'id': '195650', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 9645, 'n_tokens_neox': 9127, 'n_words': 4211, 'path': 'lectures/python-stack/u01-2-Python-Programming-data-structures.ipynb', 'repo_name': 'tgrasty/CSCI574-Machine-Learning', 'size': '30673'}