<jupyter_start><jupyter_text>Python part 3  Objects (*things*) and functions So far in Python we've come across things that exist such as:- a variable with a value: `x = 1`- a string: `s = "hello"`- a list: `a = [1,2,3]`These are examples of ***objects***: they are all separate instances of things in the computer's memory. They just exist and don't do anything on their own.We have also come across functions that ***do something***:- `print()`- `help()`- `range()`In addition to these standalone functions, all the objects in Python come with their own ***built-in functions***. We use call these functions by following the name of the object by a dot and the name of the function. For example, all lists have a `clear` function that clears the list:<jupyter_code>a = [1,2,3]
a.clear()
a<jupyter_output><empty_output><jupyter_text>As another example, all strings have a `capitalize` function:<jupyter_code>s = "hello"
print(s.capitalize())<jupyter_output>Hello<jupyter_text>Functions that belong to an object are properly called ***methods***. Every object has methods. Objects can also have variables that belong to them called ***member variables***, which we will come to later.Let's look again at list objects in Python and see what methods they have.  Data structures   Lists  We met lists in part one. We compared a string - which is a collection of characters - and a list - which is a collection of almost any object (even a mixture of objects). Let's look at some of the methods that are available to manipulate lists, by starting with a simple list and manipulating it.<jupyter_code>l = [1,2,3,4,5] # start with a simple list of ints<jupyter_output><empty_output><jupyter_text>The `append` method appends an item...<jupyter_code>l.append(6)
l<jupyter_output><empty_output><jupyter_text>...while `pop` removes one<jupyter_code>l.pop()
l<jupyter_output><empty_output><jupyter_text>`insert(i,x)` inserts object x at position i<jupyter_code>l.insert(1, 1.5)
l<jupyter_output><empty_output><jupyter_text>Clear the list with `clear`<jupyter_code>l.clear()
l<jupyter_output><empty_output><jupyter_text>Use `index` to find the index of the first occurence of an item<jupyter_code>l = ["h", "e", "l", "l", "o"]
l.index("e")<jupyter_output><empty_output><jupyter_text>and finally use `count` to count items:<jupyter_code>l.count("l")<jupyter_output><empty_output><jupyter_text>There are lots more, but these are the most commonly used ones.  Ex 3.1 The list `records` consists of personal records. Each record is simply a list with two entries: a name and a year of birth:<jupyter_code>records = [
    [ "<NAME>" , 1960 ] ,
    [ "<NAME>" , 1941 ],
    [ "<NAME>" , 1947 ],
    [ "<NAME>" , 1942 ],
    [ "<NAME>" , 1957 ],
    [ "<NAME>" , 1947 ],
    [ "<NAME>" , 1942 ],
    [ "<NAME>" , 1961 ],
    [ "<NAME>" , 1955 ],
    [ "<NAME>" , 1940 ],
]<jupyter_output><empty_output><jupyter_text>Write some code that will produce a second list that contains only those people born after 1950. Do this by:- creating an empty list;- writing a loop over each record in `records`;- checking the record, and if appropriate using the `append` method to add it to your new list.<jupyter_code>post_1950 = []
for record in records:
    if record[1] > 1950:
        post_1950.append(record)
post_1950<jupyter_output><empty_output><jupyter_text>Strings are immutable... Remember using slices with strings?<jupyter_code>s = "hello there"
print(s[0:5])<jupyter_output>hello<jupyter_text>We can do the same with a list:<jupyter_code>nums = [0, 1, 2, 3, 4, 5]
print(nums[0:3]) # first three elements
print(nums[-2:]) # last two elements<jupyter_output>[0, 1, 2]
[4, 5]<jupyter_text>We said that strings are **immutable**; they can't be altered, so you can't do<jupyter_code>#s[3] = u # you get an error if you do this<jupyter_output><empty_output><jupyter_text>... but lists *are* mutableYou can reassign individual items<jupyter_code>nums[3] = 8
nums<jupyter_output><empty_output><jupyter_text>or even slices:<jupyter_code>nums[:3] = [0, -1, -2] # reassign first three items
nums<jupyter_output><empty_output><jupyter_text>When reassigning slices like this, the sizes don't need to match:<jupyter_code>nums[:] = [1] # replace the whole list with a list of length 1
nums<jupyter_output><empty_output><jupyter_text>Ex 3.2 Given what you known of slicing, what does the following do?<jupyter_code>nums = [1, 2, 3, 4, 5]
nums[:] = list[::-1]
list<jupyter_output><empty_output><jupyter_text>Clearing bits of strings We have used slices to select part of a list. Now we can **delete** part of a list by taking a **slice** and assigning it to an empty list, `[]`.Clear the first two items:<jupyter_code>nums = [1,2,3,4,5]
nums[0:2] = []
nums<jupyter_output><empty_output><jupyter_text>clear the whole list:<jupyter_code>nums[:] = []
nums<jupyter_output><empty_output><jupyter_text>clear a single item<jupyter_code>nums = [1,2,3]
nums[0:1] = []
nums<jupyter_output><empty_output><jupyter_text>Notice that we have to select a slice to replace, rather than an item. This doesn't work:<jupyter_code>nums[0] = []
nums<jupyter_output><empty_output><jupyter_text>as we end up with a list within a list.  Tuples Tuples are very like lists. A tuple (usually pronouced "tyou-pell" but sometimes "tupple") is a collection that- may contain different object types- is ordered- is ***immutable*** - this is probably the key difference.We define a tuple like a list, but without the square brackets:<jupyter_code>tuple = 1, 2, "a", 2.4
tuple<jupyter_output><empty_output><jupyter_text>A tuple of length one is defined using a trailing comma:<jupyter_code>notatuple = 0
isatuple = 0,
type(isatuple) # check it's a tuple<jupyter_output><empty_output><jupyter_text>Sometimes round brackets are used to enclose a tuple, though they're not necessary.A nice use of tuples is to assign multiple variables at once:<jupyter_code>x, y, z = 1, 2, 3
print(x, y, z)<jupyter_output>1 2 3<jupyter_text>This statement really containts two tuples - `x, y, z` and `1, 2, 3` - and we are able to assign individual elements separately.You can access elements in a tuple as you would a list:<jupyter_code>tuple[0]<jupyter_output><empty_output><jupyter_text>Ex 3.3 Create a tuple containing a string, an int, a float and a list.<jupyter_code>tuple2 = ("hello", 7, 3.4, [1, 2] )<jupyter_output><empty_output><jupyter_text>Sets Sets are collections that- are unordered- have no duplicate itemsDefine a set using brace brackets:<jupyter_code>set = {"h", "e", "l", "l", "o"}
set<jupyter_output><empty_output><jupyter_text>Notice:- the set doesn't let us have duplicates (the second l was removed)- the order is disregarded Check the presence of an item using `in`:<jupyter_code>"h" in set<jupyter_output><empty_output><jupyter_text>Add an item to the set using `add`...<jupyter_code>set.add("z")
set<jupyter_output><empty_output><jupyter_text>...and remove one using `remove`<jupyter_code>set.remove("e")
set<jupyter_output><empty_output><jupyter_text>Some methods are similar to those for lists, such as `clear`<jupyter_code>set.clear()
set<jupyter_output><empty_output><jupyter_text>Dictionaries The list is a data structure that allows you to access a value - an item in the list - using an index, i.e. its position:<jupyter_code>students = ["Alice", "Chris", "Pavel", "Pablo"]
students[1]<jupyter_output><empty_output><jupyter_text>A dictionary allows you to access a ***value*** using a ***key*** rather than an index. Let's use a dictionary of students to store their grades:- the ***key*** is their **name**- the ***value*** is their **grade**<jupyter_code>students = {"Alice": 13, "Chris": 11, "Pavel": 19, "Pablo": 4, "Martin" : 13}
students<jupyter_output><empty_output><jupyter_text>Each item in the dictionary is defined using **key**:**value**, and the whole thing is in brace brackets.A dictionary:- has no duplicate *keys* (just like a paper dictionary has no duplicate words)- can have identical *items* (as long as they have a different key): Alice and Martin have the same grade as we saw- is unordered - we don't care about the position of itemsAccess an item using its key:<jupyter_code>students["Alice"]<jupyter_output><empty_output><jupyter_text>Add an item<jupyter_code>students["Aziz"] = 12
students<jupyter_output><empty_output><jupyter_text>Remove an item using `pop`<jupyter_code>students.pop("Aziz")
students<jupyter_output><empty_output><jupyter_text>Check for a key using `in` (in the same way as we did with a set):<jupyter_code>"Alice" in students<jupyter_output><empty_output><jupyter_text>Ex 3.4 An encrypted message has been discovered together with part of a codebook describing the simple cipher used to encrypt it. For each entry in the dictionary below, the key is a letter in the secret message, and the value is a letter in the plaintext original message:<jupyter_code>message = "shp#hars#rtdw#oitlw#halw#izwa#shp#xmw#yhma#xak#halw#izwa#shp#rhhj#kwxoz#ta#ozw#bxlw"

codebook = {
# code : original
 'y': 'b',
 'l': 'c',
 'k': 'd',
 'b': 'f',
 'f': 'g',
 'z': 'h',
 't': 'i',
 'n': 'j',
 'j': 'k',
 'r': 'l',
 'q': 'm',
 'a': 'n',
 'h': 'o',
 'v': 'p',
 'g': 'q',
 'm': 'r',
 'u': 's',
 'p': 'u',
 'd': 'v',
 'i': 'w',
 'c': 'x',
 's': 'y',
 'e': 'z',
 '#': ' '
}<jupyter_output><empty_output><jupyter_text>However, the last three crucial entries are missing. These have been discovered to be- `'w': 'e'`- `'o': 't'`- `'x': 'a'`- Add these entries to `codebook` like we did with the entry "Alice". - Write a loop that takes each letter of `message` and uses `codebook` to look up the unencrypted letter, to decipher the message.  Functions `list`, `tuple`, and `set` We have seen how to define collections - lists tuples and sets - already:<jupyter_code>my_list = [1, 1, 2, 3]
my_tuple = (1, 1, 2, 3)
my_dict = {"a" : 1 , "b" : 1 , "c" : 2, "d" : 3}<jupyter_output><empty_output><jupyter_text>The functions `list`, `tuple` and `set` allow us to define collections another way, by turning one sort into another. For example, let's turn a tuple into a list:<jupyter_code>new_list = list(my_tuple)
new_list<jupyter_output><empty_output><jupyter_text>or vice-versa:<jupyter_code>new_tuple = tuple(my_list)
new_tuple<jupyter_output><empty_output><jupyter_text>Turn a list into a set (and notice the duplicate entry is removed)<jupyter_code>new_set = set(my_list)
new_set<jupyter_output><empty_output><jupyter_text>These functions will work with other iterable objects (i.e. things you can iterate over). Let's try with a string (which is just a collection of characters after all):<jupyter_code>agent = "saunders"

string_list = list(agent)
print(string_list)

string_tuple = tuple(agent)
print(string_tuple)

string_set = set(agent)
print(string_set)<jupyter_output>['s', 'a', 'u', 'n', 'd', 'e', 'r', 's']
('s', 'a', 'u', 'n', 'd', 'e', 'r', 's')
{'e', 'n', 'r', 'u', 's', 'a', 'd'}<jupyter_text>Ex 3.5 Given a string `words`, write code that creates a list of the unique characters in that string. For an extra challenge, create a dictionary where every key is a letter in the string, and each value is the number of occurences of that letter. E.g. the word "salsas" would produce the dictionary `{"s" : 3, "a" : 2, "l" : 1}`. *Hint:* the `count` function counts the number of occurrences in a string.<jupyter_code>words = "natalia simonava"
letter_list = list(set(words))
print(letter_list)

letter_dict = {}
for letter in letter_list:
    letter_dict[letter] = words.count(letter)
letter_dict<jupyter_output>['i', 'v', 'n', 'l', 'm', ' ', 'a', 's', 'o', 't']<jupyter_text>Comprehensions As stated in the introduction, the intention with these tutorials is to explain the basic elements of Python to allow you to write code quickly, without having to invest in learning some of the more advanced language features. Comprehensions are one such advanced feature that we *are* going to consider, because:- they provide an elegant syntax which will spare you a lot of typing when solving common problems;- you will see them all the time in Python code examples, so it makes sense to understand them.Here is the motivation: an extremely common task is that of creating a list programmatically - let's take a list of fractions as an example:$\left[\frac{1}{1}, \frac{1}{2}, \frac{1}{3}, \cdots, \frac{1}{10}\right]$.Using our knowledge we could code this like so:<jupyter_code>fractions = []
for i in range(1, 11):
    fractions.append(1 / i)
fractions<jupyter_output><empty_output><jupyter_text>The code above takes an empty list, and uses a range of values. For each value in the range, it calculates something (a fraction in this case) and adds it to the list. This task is so common that Python provides a neat syntax for it, called a ***list comprehension***:<jupyter_code>fractions = [1 / x for x in range(1, 11)]
fractions<jupyter_output><empty_output><jupyter_text>Here is another example: create a list of all the individual letters in a string:<jupyter_code>letters = [l for l in "i never joke about my work 007"]
letters<jupyter_output><empty_output><jupyter_text>We see that a list comprehension has the form:[*do something* with *VARIABLE* `for` VARIABLE `in` *OBJECT you can iterate over, e.g. list, range, string*].Here is another example that takes a list of words and produces a list of their lengths:<jupyter_code>quote = ["Pistols", "at", "dawn", "it's", "a", "little", "old", "fashioned", "isn't", "it"]
[len(word) for word in quote]<jupyter_output><empty_output><jupyter_text>Suppose now we want to count words lengths again, but we want to ignore any instances of the words "a" and "the". Fortunately, Python allows you to add an if statement at the end of a list comprehension that will determine whether or not the element is included in the new list:<jupyter_code>quote = ["a", "cat", "sat", "on", "the", "mat"]
[len(word) for word in quote if word != "a" and word != "the"]<jupyter_output><empty_output><jupyter_text>Ex 3.6 Write a list comprehension that produces a list of the squares $x ^ 2$ for $x$ in $1, 2, 3, \cdots, 100$, but only if $x ^ 2$ is a multiple of 3 and 5.<jupyter_code>squares = [x ** 2 for x in range(1, 101) if (x ** 2) % 3 == 0 and (x ** 2) % 5 == 0]
squares<jupyter_output><empty_output><jupyter_text>Assiging variables and variable bindingRemember that we can assign variables to other variables, so we can create a variable `x` and then assign its value to `y`:<jupyter_code>x = 1
y = x<jupyter_output><empty_output><jupyter_text>If we alter one, the other stays unchanged:<jupyter_code>x += 1 # increment x: check y remains unchanged
print(x, y)<jupyter_output>2 1<jupyter_text>If you try the same thing with a list, something strange happens:<jupyter_code>list_x = [1, 2, 3]
list_y = list_x
list_y.clear()
print(list_x, list_y)<jupyter_output>[] []<jupyter_text>Clearing `list_y` has had an effect on `list_x`! In fact the same will happen for any object that is mutable, e.g. a set:<jupyter_code>set_x = {1, 2, 3}
set_y = set_x
set_y.remove(1)
print(set_x, set_y)<jupyter_output>{2, 3} {2, 3}<jupyter_text>This is because of the way assignment works in Python. What we actully are doing when entering `x = 1` is creating an association between the name `x` and the object `1`: we say that the object `1` is bound to `x`. Now when we do `y = x`, the variable `y` is also bound to whatever `x` is bound to (`1`). We ***do not copy*** `y` into `x`.When we increment `x` with `x += 1`, we don't alter the object that `x` is bound to (it's a number which is immutable); rather, `x` is reassigned to the object `2`. The picture below explains this. Every object in Python has a unique ID. By checking this ID with the `id` function, we can show that when we perform assignment, the two variables refer to the same object, rather than a copy. Let's do this with strings:<jupyter_code>x = "abc"
x = y
print(id(x), id(y))<jupyter_output>140736851603200 140736851603200<jupyter_text>Now it's clear what happened with the lists: `list_x` and `list_y` actually both refer to the same object; we can check:<jupyter_code>print(id(list_x), id(list_y))<jupyter_output>2184141071944 2184141071944<jupyter_text>Since lists are mutable, any changes to `list_x` or `list_y` change a single underlying object. The picture below explains this. So we see that:***assigning variables simply means binding them to an object; assigning one variable to another means that both will refer to the same underlying object.***  Altering things inside a function We have just seen that if we perform an assignment of the form  `x = y` then:- if `x` and `y` are mutable objects, any change in `x` is reflected in `y`;- if `x` and `y` are numbers, then alterations to `x` don't affect `y`.Similarly, we can think about how variables are changed when they are passed in to a function. The following function takes a value `number` and a list `list` and alters each one:<jupyter_code>def alter(number, list):
    number += 1 # add one to number
    list.append(1) # append a 1 to list<jupyter_output><empty_output><jupyter_text>Let's define some variables, and see how they look before and after passing them to our function:<jupyter_code>my_number = 0
my_list = []

print("before function")
print (my_number, my_list)
alter(my_number, my_list)
print("after function")
print(my_number, my_list)<jupyter_output>before function
0 []
after function
0 [1]<jupyter_text>Just like earlier when we investigated assigning variables to each other, the number is left unchanged, but the list is altered.It's quite useful being able to pass an argument to a function and have the function alter it *in place*, rather than having to return a value. As an example, here is a function to swap a list containing two items:<jupyter_code>def swap(l):
    l[0], l[1] = l[1], l[0]

# testing
l = [1,2]
swap(l)
l<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12115427801482147, 'fraction_numerical': 0.017628565012351223, 'mean_word_length': 5.5344827586206895, 'pattern_counts': {'":': 4, '<': 239, '<?xml version=': 0, '>': 240, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '60', 'dir': 'jupyter-structured-clean-dedup', 'id': '489512', 'license': 'MIT', 'n_tokens_mistral': 6241, 'n_tokens_neox': 5914, 'n_words': 2271, 'path': 'part_3.ipynb', 'repo_name': 'stuwilmur/begin-python', 'size': '17879'}