<jupyter_start><jupyter_text>BK3OV3         Python, Rhino and Grasshopper        Part 2 : Data structures        &copy; 2019, TU Delft. <NAME>, Creative Commons                      Data structuresIn Part 1 we used some basic data types (scalars), like integers and floats, and we mentioned *strings*, a non-scalar data type.Python has some other more complex data types which have a certain *structure*. One of them was already briefly mentioned, the *string*. A data structure is capabale of storing different data types and even data structures. In Python data structures are *objects*, we will explain what objects are later, that have some fancy capabilities (*methods*, will also be explained later).Python discerns five different data "categories" and each category has certain data types:CategoryTypeDescriptionNoneNoneNull objectNumbersintIntegersfloatFloating pointcomplexComplex numberboolBoolean, true or falseSequencestrString, characterslistList of datatupleTuplerangeRange of integersMappingdictDictionarySetssetMutable setfrozensetImmutable setIn this notebook we will discuss lists, ranges, dictionaries and tuples. In the next notebook (Part3) we will discuss strings. ListsLet's first start with *lists*, probably the most used data type/structure. A list can be seen exactly the same way as a list in your notebook, you have lines and on those lines you write your to-do things or doodles. You define a list the following way:<jupyter_code>a = []
b = list()
c = ["egg", "spam", "lard"]<jupyter_output><empty_output><jupyter_text>On the first line we define an empty list by using the brackets, on the second line we use the `list` function and on the last line we create and *initialize* (give values to) a list. We seperate the different values with commas. We can add elements to a list by using the `append` method:<jupyter_code>a.append("cat")
print(a)
a.append("dog")
print(a)<jupyter_output><empty_output><jupyter_text>To retrieve an element from a list you specifiy the index of the element between brackets (see it as referecing a line number in your notebook). It is worth noting that in the world of programming 0 is the first element of a list, thus:<jupyter_code>a[0]<jupyter_output><empty_output><jupyter_text>You can also use negative numbers, in which case you start counting from the end of the list. -1 is the last element in a list:<jupyter_code>print(c[-2])
print(c[1])<jupyter_output><empty_output><jupyter_text>Can you figure out the counting in the cell above? -1 is the end so -2 is the... Let's make another list of some other animals and then add this new list to list _a_ and assign this list to *c*:<jupyter_code>b = ["cow", "chicken", "hippo"]
c = a + b
c<jupyter_output><empty_output><jupyter_text>With `append` you can only add one element to a list. To add two lists together we can use the add operator (+). That makes sense right? Another option is by using the *extend* method: `c.extend(b)`. If we want to know how many cows we have, we can use the `count` method. If we want to know the position of the *first* occurence of cow in the list, we can use the `index` method.<jupyter_code>num_cows  = c.count("cow")
first_cow = c.index("cow")
print("There are", num_cows, "cow(s) in the list and the first one is at index:", first_cow)<jupyter_output><empty_output><jupyter_text>Our cow is a bit lonely so let's add another one. Cows like to be close together but not too close so we want the chicken to separate them. We can find the location of the chicken with the `index` method. Then we need to insert a new cow with the `insert` method. Once we have done that we need to check that there are actually two cows.<jupyter_code>place_cow = c.index("chicken")
# we add one to place the cow AFTER the chicken
c.insert(place_cow + 1, "cow")
num_cows  = c.count("cow")
first_cow = c.index("cow")

print("There are", num_cows, "cow(s) in the list and the first one is at index:", first_cow)<jupyter_output><empty_output><jupyter_text>We have now the following animals in our list:<jupyter_code>c<jupyter_output><empty_output><jupyter_text>The cat has been chased away by the dog and replaced by a hamster. We can simply assign a new value to an element in the list by using the square brackets and an index number:<jupyter_code>c[0] = "hamster"
print(c[0])<jupyter_output><empty_output><jupyter_text>Unfortunately, the hamster was trampled by one of the cows. We need to remove the hamster from our list. We can delete an element from a list with `remove` method or with a `del` statement:<jupyter_code># remove the hamster
c.remove("hamster")

# remove the dog, because he was nasty to the cat
del c[0]
c<jupyter_output><empty_output><jupyter_text>You see that `del` is neither a method - we don't use the data type with a . (dot) - nor a function - because we don't do `del(a[0])`.Another way to delete an element is to use the `pop` method. The difference between the remove and del approach is that pop will return the removed element. If you use pop without an argument the last element of the list is retrieved:<jupyter_code>chick = c.pop(1)
hyp   = c.pop()

print("The", chick, "and the", hyp)<jupyter_output><empty_output><jupyter_text>We are now left with only cows which is boring so let's repopulate our list again and reverse the order of the list.<jupyter_code>c = a + b
print(c)

c.reverse()
print(c)<jupyter_output><empty_output><jupyter_text>For loopsA lot of times you want to repeat a certain block of code. For example, you want to express your dedication to your local football team by shouting "Ajax" or "Feyenoord". Suppose you want to do this 9 times after which you need to take a breath. You could write a `print("Ajax!")` and then copy paste that 9 times. What if you decide half way the match to switch sides because Pec Zwolle is the better team? You have to change nine print statements, not very smart. Or instead of printing the whole list of animals on one line, you want to print the name of the animal on a seperate line.To iterate a certain number of times or to iterate over the elements of a list Python has the `for` loop construct. The for loop has the following syntax:>for _variable_ in _sequence_:>&emsp; _do something_ variable is the iteration variable, sequence is the data structure or we want to iterate overthe for ... in will return a new value to _item_ from the sequence _sequence_ with each iteration roundIn case of the football example we want to print our expression of idolation 9 times:&emsp;`for i in sequence:`&emsp;&emsp;`print("Ajax!")`To do something nine times we probably need a range of numbers. We could make a list with nine numbers but Python has a _range_ data type that does just that.  RangeIn the table at the beginning of this notebook you can see that in the type category of sequence we have a range data type. The _range_ data type gives us a range of integers. You define the following way:<jupyter_code># range(start number, end number, steps)
# end number is not included in the sequence
# and the steps parameter is optional
numbers = range(0, 10)
even = range(0, 10, 2)
print(numbers) #! That is probably an unexpected output
print(numbers[3])
print(len(numbers))<jupyter_output><empty_output><jupyter_text>The `len` function returns the numbers of elements in a sequence.With the `for` loop construct we can now print the numbers in the sequence _numbers_.<jupyter_code>numbers = range(0, 4)
for i in numbers:
    print(i)<jupyter_output><empty_output><jupyter_text>You van write the code above more compact. Notice we also don't specify the start number. Python will automatically assume that we want to start the sequence at the number 0.<jupyter_code>for i in range(4):
    print(i)<jupyter_output><empty_output><jupyter_text>We can now finish the code to show our dedication to Ajax:<jupyter_code>for i in range(9):
    print("Ajax!")<jupyter_output><empty_output><jupyter_text>The in operandIn the code above you see that _in_ is green colored and bold. It is thus a Python keyword. What does it do? It checks for membership:<jupyter_code>got_hippos = "hippo" in c
print(got_hippos)

if "camels" in c:
    print("We got camels.")
else:
    print("There are no camals.")<jupyter_output><empty_output><jupyter_text>In the examples above we got a value from the range sequence in our variable i. If we use the list with our animals, we get an element from that list in i:<jupyter_code>for i in c:
    print(i)
    print(type(i))
# print all animals in uppercase
for i in c:
    uppercase = i.swapcase()
    print(uppercase)<jupyter_output><empty_output><jupyter_text>FunctionsI am pretty sure some of you didn't like the Ajax idolation code. You probably want to cheer for another team or maybe cheer for something that is not related to football at all. Suppose you are the cheery type and like to cheer for a lot of things, you would end up retyping the same lines of code over and over. Whenever you need to retype code you should consider making a function of it. A function is a block of code you can You define a function with the keyword `def` followed by a descriptive function name. Then you might want to have inputs to your function, these are called parameters. You specify parameters by placing variable names, seperated by commas, between parentheses after the function name. You finish the function defenition by ending it with a colon, like:>`def myFunction(input1, input2):`Then you make an indentation and write your code. To use your function you write the name of the function and you give values to the variables. We could write an idolation function called cheer for example:<jupyter_code>def cheer(yell, x_times):
    for i in range(x_times):
        print(yell)
        
cheer("I like Python!", 3)<jupyter_output><empty_output><jupyter_text>The _cheer_ function does not return a value. In the world of programming this is called a _void_ function. If we want to create a function that returns a value/data we use the __return__ keyword followed by the data we want to return:<jupyter_code>def moment(load, length):
    m_max = load * length**2 / 8.0
    return m_max

load = 3
length = 7
max_moment = moment(load, length)
print(max_moment)<jupyter_output><empty_output><jupyter_text>ScopeIn the code above I use the name load four times, in the main body of the code as variable, as input for the function call, as parameter in the function definition and then in the function code. The variables defined outside the function, in the main body of the code, are not the same as the variables within the function though they have the same name.It can be better explained with a more straightforward example. In the code in __example 3__ below we define a variable b and assign it the value 4. We define a function with a parameter called b. When we call the function scope with the variable b, it is the variable from the outer code block (all the code at the left side of the cell). The b in the function call is not the same as the one in the function definition. The parameter b in the function call gets replaced with the value of variable b (4). In the first print statement b has the value of the b from outside the function (4). Then we change the b inside the function and print it (7). When the function is done executing we continue with the main body code again and print b from the main body code, which is still 4.This is called the scope of a variable. This means that a variable or name only has a certain area where it is valid. Variables defined within a function or object only "live" within that function or object, they are __local__ variables (This behaviour also holds true for variables in external python modules). Variables defined in the main body code are called __global__ variables.  Example 1:<jupyter_code>b = 5

def scope():
    b = 7
    print(b)
    
scope()
print(b)
# we didn't pass b from the main body to scope
# the b in the scope function is locally defined<jupyter_output><empty_output><jupyter_text>Example 2:<jupyter_code>b = 4

def scope():
    print(b)
    
    
scope()
print(b)
# we didn't pass b from the main body to scope
# since there is no b inside the scope function 
# it can access the globally defined b<jupyter_output><empty_output><jupyter_text>Example 3:<jupyter_code>b = 4

def scope(b):
    print(b)
    b = 7
    print(b)
    
scope(b)
print(b)
# we have a parameter in our function, it gets a value
# when we call the function. The b inside the function
# is a local variable and doesn't change b outside of
# the function block<jupyter_output><empty_output><jupyter_text>List slicingList slicing is a way to extract a part from a list. We first make our list of animals a bit longer so we can slice it up (the list, not the animals!):<jupyter_code>zoo = c + ["zebra", "cobra", "mouse", "gnu", "lamb"]
print(zoo)<jupyter_output><empty_output><jupyter_text>Slice operations work as follows:+ you give the name of the list,+ you specify the start index,+ you specify the end index (the end is exluded so it is up to the end),+ and you optionally specify the number of steps you want to take,+ all these numbers/expressions are seperated by a :so:`zoo[1:10:4]`<jupyter_code># prints the second and third element
print(zoo[1:3:])
# print the elements starting at the second
# index and then take steps of 3 indices
print(zoo[1:9:3])<jupyter_output><empty_output><jupyter_text>The second print statement works as follows. The first element at index 1 is chicken. Then we take three steps which gives us the cat. We take three more steps and we get the cow. If we take three more steps we reach element 10 and that falls outside of the range we have set, namely 9.If we omit the start, element 0 is presumed, if we omit the end then the end of the list is presumed.<jupyter_code>print(zoo[:3])
print(zoo[8:])

# this does the same as above but counting backwards
print(zoo[-2:])<jupyter_output><empty_output><jupyter_text>Advanced list methodsSo far we have been adding only strings to our list data structure. As mentioned above we can add data structures to data structures. Let's make a new list with some predators, assign this new list to the element with the value chicken and then print it.<jupyter_code>predators = ["wolf", "cheetah", "owl"]
chicken_index = c.index("chicken")
c[chicken_index] = predators

for i in c:
    print(i)<jupyter_output><empty_output><jupyter_text>If we want to access an element inside the list in a list we use the following syntax:<jupyter_code>print(c[1][2])<jupyter_output><empty_output><jupyter_text>You should read the _c\[1\]\[2\]_ from right to left: so first get the c\[1\] element and then from that element get element 2.You see that the for loop in cell 25 prints the list we have just inserted as a...list. If we want to neatly print all the animals beneath each other we need to come up with another solution. If *i* is a list and not a string or number (either int or float) then we want to print the elements of that list again. So first we need to figure out what kind of data type or structure _i_ is. For that we can use the function `type`.<jupyter_code>print(type(c))
print(type(2))
print(type("a string"))
type(c) == type([])<jupyter_output><empty_output><jupyter_text>>  intermezzo 1: ObjectsOn the first three lines you can see the output of the type function for some data types. Now below cell 1 we mentioned *objects*. In Python everything is an object. When we create a new variable, either a simple data type or a data structure, we create an *instance* of a *class*. A class can be seen as the skeleton or blueprint (or a production drawing to stay a bit closer to home) of an object. So we see that c is an object of the list class and 2 is an object of the int class.On the last line we compare c, using the double equal sign, with an empty list (the \[ \]). Comparisons return a boolean value, either `True` or `False`. Since _c_ is a list and an empty list is also a list the comparison returns `True`. So now that we can check what data types we have, we can do the following:<jupyter_code>if(type(c) == type([])):
    for i in c:
        print(i)<jupyter_output><empty_output><jupyter_text>RecursionWe get the same result...So inside the for loop we need to copy the code again. But what if we have a list in a list in a list? Do we then have to copy paste the code inside the code inside the code etc? That's not a very smart way to tackle this puzzle. We duplicate code so we should start thinking about writing a function. If we want to print the items of a list we could write the following function:>`def printList(a_list):    for i in a_list:        print(i)`This doesn't solve the problem of a list in a list. We should do a check if at a certain moment _i_ will be a list. If _i_ is not a list we can just print the value. So our improved function might look like:>`def printList2(a_list):    for i in a_list:        if (type(i) == type([])):            do_something        else:            print(i)`What code should be on the line *do\_something*? We have a function that has as argument/input a list. If an element in this list is **not** a list we want to print it - which is what we do in the *else* block. If an element **is** a list we want to run this same function again. We pass this list to *printList2* again and then *printList2* will check if all the elements of this list are now just normal elements if it contains maybe another list, in which case, we call *printList2* again. This programming trick is called *recursion*. So now we can write our final version:<jupyter_code>def printNestedList(a_list):
    for i in a_list:
        if (type(i) == type([])):
            printNestedList(i)
        else:
            print(i)
            
# call printNestedList with the c list
printNestedList(c)<jupyter_output><empty_output><jupyter_text>Hurrah, it works! You probably still remember, but this function is a void function, it doesn't return a value. Enumerating and  zippingBesides the `for i in s:` construct there are (two) more ways to iterate over a list using the `enumerate` and `zip` function. The enumerate function returns two values, one being the current iteration number, the other the element in the list at the current iteration index:>`for i, x in enumerate(s):`With the `zip` function you can iterate over two or more lists in parallel, taking an item of each of the lists:>`for x, y in zip(s, t):`Examples:<jupyter_code>first_list  = ["brick", "mud", "wall", "beam"]
second_list = ["beam", "mud", "wall", "brick"]

for i, x in enumerate(first_list):
    print(i, ":", x)

#print an empty line
print()

for x, y in zip(first_list, second_list):
    print(x == y)<jupyter_output><empty_output><jupyter_text>DictionariesAnother basic data structure in Python is the dictionary. As a normal dictionary, you look up some index (a difficult word) and you get the value (hopefully a simple explanation of the difficult word). You create a dictionary as follows:<jupyter_code>empty_dict = {}
also_empty = dict()
type(also_empty)<jupyter_output><empty_output><jupyter_text>You notice that it doesn't use the brackets we used with lists, but braces ({). Of course you can initialize a dictionary as well:<jupyter_code>my_dict = {"first" : 0, "second" : 1, 34 : 45}
my_dict<jupyter_output><empty_output><jupyter_text>So a dictionary consists of a *key* and a *value*, which are seperated from each other with a colon (:) and the different key-value pairs are seperated by a comma (,). For a key you don't have to use strings, you can use other Python objects like numbers and tuples (very simple lists, more about those later) though you can not use lists and dictionaries as keys. As a rule of thumb, you can not use objects which can change - which are mutable - as keys. If this doesn't make sense, don't worry, it will be explained later.You can assign a value to a key or retrieve a value from a key the same way as with lists. You use square brackets now (and yes that could be considered confusing).<jupyter_code>print(my_dict["first"])
my_dict['second'] = 9
print(my_dict['second'])
print(my_dict[34])<jupyter_output><empty_output><jupyter_text>Some remarks: 1. It doesn't really matter if you use a single quote or a double quote for the key if the key is a string.2. The type of the key does matter: \["34"\] will not work nor will \[second\] work.To get all the keys or all the values in a dictionary you can use the methods `keys` and `values`:<jupyter_code>my_dict.keys()
my_values = my_dict.values()

# my_values is a data structure resembling a list
print(type(my_values))
print(my_values)

for i in my_values:
    print(i)

# but this does not work
print(my_values[0])<jupyter_output><empty_output><jupyter_text>To get all the items in a dictionary as a list of tuples you can use the `items` method. To test if a dictionary has a certain key you can use the `in` operator:<jupyter_code>print(my_dict.items())

for i in my_dict.items():
    my_key, my_value = i
    print("The key is", my_key, "and the value is", my_value)

#print the number of elements in my_dict
print("Length of my_dict:", len(my_dict))

# check if we have a key "second" in my_dict
print("second" in my_dict)

# check if we still have a hamster in c -> False because a cow trampled it
print("Do we have a hamster in c?", "hamster" in c)

# check if we have a cow
print("Do we have cows then?", "cow" in c)<jupyter_output><empty_output><jupyter_text>There are many more methods associated with lists and dictionaries. An easy way to find what kind of methods an object has is to use the `dir` function with the object as parameter:<jupyter_code>dir(my_dict)<jupyter_output><empty_output><jupyter_text>We can do the same with any Python libraries that we import:<jupyter_code>import math
dir(math)<jupyter_output><empty_output><jupyter_text>TuplesSo the last object we'll visit is the tuple. It looks like a list but it works a little bit differently. In the code in cell 28 you saw the line reading:>`my_key, my_value = i`If you print the type of i you will see that it is a of type tupple. If you print i you will get something like ('my text', 'more text', 3, 'a'), that is a comma seperated list of elements between parentheses. Tuples are used, among others, as an easy way to return more then one value from a function; for example:def multi_ret(a, b):    return (a * b, a + b)(result1, result2) = multi_ret(4, 6)<jupyter_code>my_tuple =('cat', 'mouse')
print(type(my_tuple))
print(my_tuple[0])

# concatenation works
new_tuple = my_tuple + my_tuple
print(new_tuple)<jupyter_output><empty_output><jupyter_text>After you create a tuple you cannot add any more variables or data to it. In Python they call this *immutable* data type.The tuple data type has a very limited set of methods:<jupyter_code>dir(my_tuple)<jupyter_output><empty_output><jupyter_text>A little bit more about objectsLet's create a new list, assign that list to a variable and then assign that variable to another variable. When we print the variables we'll see they are the same (of course, duh...).<jupyter_code>prog_lang = ["java", "c++", "algol"]
more_lang = prog_lang
print(prog_lang, "\n", more_lang, sep='')<jupyter_output><empty_output><jupyter_text>> Intermezzo 2: Escape charactersIn the print command you see something funny (hopefully...) which is the *\n* and the *sep=''* (two single quotes). The *\\n* is an *escape character* and creates a new line (an enter on your keyboard). Some other escape characters are:+ \a: bell+ \v: vertical tab+ \t: horizontal tab>>Sometimes when you have a string and you want to have a double quote inside the string you need to tell the print statement that it shouldn't treat the double quote as a string termination. You do this by *escaping* the character with a backward slash \\, for example:<jupyter_code>print("This is a \" double quote")
print("This: " does not work")
print("Nor does this \")
# escaping the escape character
print("But this does \\")<jupyter_output><empty_output><jupyter_text>Put a  in front of the line to disable it. The code in the cell will not be displayed until all errors are fixed.> Intermezzo 3: HelpThen we have the *sep=''*. This tells the print command to not use a space as a seperator between the arguments (inputs). You can remove it from the print statement to see what happens... A lot of the time while programming, things go wrong or not the way you wanted (one might say that this is the case 99% of the time and not only with programming). So you have to read the manual or search online ([Stackoverflow](https://stackoverflow.com/questions/tagged/python)).If you are using functions in Python you can use the `help` function to print help about a **function** or  **module** (and yes, you can do `help(help)`).<jupyter_code>help(print)

#help(help)<jupyter_output><empty_output><jupyter_text>Let us continue with our two lists. Change one of the elements of list prog_lang and then print the list more_lang.<jupyter_code>prog_lang[0]="R"
print(more_lang)<jupyter_output><empty_output><jupyter_text>You see that they are both the same again! When can check this in two other ways using the compare operator `==` and the `is` operator<jupyter_code>print(prog_lang == more_lang)
print(prog_lang is more_lang)<jupyter_output><empty_output><jupyter_text>The first statement test if prog_lang and more_lang have the same element values; apparently they do! How is this possible, because we only changed prog_lang and not more_lang! The second print statement gives us a hint. The `is` operator test if two variables or objects to see if the refer to the same object in memory. Let's revisit that very verbose sentence:>Let's create a new list, assign that list to a variable and then assign that variable to another variable. What happens here is that the list we write in the cell is the first object that get's created by Python. Then we assign that object to a variable called prog_lang. So prog_lang refers to the list which is somewhere in the memory of our computer, prog_lang points to the memory address of our list. If we now assign prog_lang to more_lang, we basically point more_lang to the same address in the computer's memory.\If we make a new list, with the exact same elements as our prog_lang list, assign it to a new varaible and then compare it with prog_lang, we see that it returns `True` when we compare it for *equality* but `False` when we compare it for *identity*.<jupyter_code>other_lang = ['R', 'c++', 'algol']

print(prog_lang == other_lang)
print(prog_lang is other_lang)<jupyter_output><empty_output><jupyter_text>Below you see the behaviour of some Python objects:<jupyter_code>import copy

# compare integers and numbers
print("Numbers:")
wn = 1
tn = 1
print(tn == wn)
print(tn is wn, "!")
# but if you change t, w won't change

# compare tuples
print("\nTuples:")
at = ("a", "b")
bt = at
print(bt == at)
print(bt is at)

# making a deepcopy of a tuple does....
print("\nTupple deepcopy:")
b = copy.deepcopy(a)
print(b == a)
print(b is a, "!")

# with strings
print("\nStrings:")
ast = "Some text."
bst = ast
print(bst == ast)
print(bst is ast)
print("\nStrings deepcopy:")
cst = copy.deepcopy(ast)
print(cst == ast)
print(cst is ast)

# making a deepcopy of a list or dictionary does...
print("\nList and dict:")
prog_lang = ["java", "c++", "algol"]
prog_dict = {"java" : "<NAME>", "C++" : "<NAME>", "Python": "<NAME>"}
more_lang  = prog_lang
more_dict = prog_dict
print(prog_lang == more_lang)
print(prog_lang is more_lang)
print(more_dict is prog_dict)

print("\nList and dict deepcopy:")
more_lang = copy.deepcopy(prog_lang)
more_dict = copy.deepcopy(prog_dict)
print(prog_lang is more_lang)
print(more_dict is prog_dict)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0765572648340959, 'fraction_numerical': 0.004431314623338257, 'mean_word_length': 4.735123966942148, 'pattern_counts': {'":': 2, '<': 197, '<?xml version=': 0, '>': 212, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '48', 'dir': 'jupyter-structured-clean-dedup', 'id': '533697', 'license': 'MIT', 'n_tokens_mistral': 8225, 'n_tokens_neox': 7947, 'n_words': 4082, 'path': 'notebooks/BK3OV3_part2.ipynb', 'repo_name': 'shervinazadi/earthy_workshops', 'size': '27794'}