<jupyter_start><jupyter_text>Containers and Control FlowThis lesson will introduce you to Python's most common data stuctures, i.e. **containers**. As the name implies, containers are objects for holding collections of other objects.As we learn about containers, we'll also encounter some new techniques that help us better control code execution. These techniques fall under the umbrella term **control flow** and include concepts like iteration, conditional statements, functions, and error handling.  Lesson ObjectivesBy the end of this lesson, you will be able to:- Construct some common container types, including lists, sets, dictionaries, and tuples- Iterate through these containers- Write functions to utilize code in more than one place in a program- Understand errors and handle exceptions  Table of Contents - [Containers](containers)     - [Lists](list)     - [Sets](set)     - [Tuples](tuple)     - [Dictionaries](dict) - [Functions](functions) - [Errors and Exceptions](errors) - [What's Next](next) - [The Star Wars API:  Parsing JSON Using Lists & Dictionaries](starwars)  ContainersYou've already worked with one type of container:  strings. This may not be readily apparent until you think of a word, like `spam`, as nothing more than a container for the letters `s`, `p`, `a`, and `m`:<jupyter_code>spam = 's' + 'p' + 'a' + 'm'
spam<jupyter_output><empty_output><jupyter_text>Just like the string above, containers are objects that hold *sequences* of other objects. Think of them as data structures that live in memory and also (typically) hold their values in memory. Since containers hold objects that live in memory, containers support membership tests with the keywords `in` and `not in`:<jupyter_code>'s' in spam
'ham' not in spam<jupyter_output><empty_output><jupyter_text>In this lesson, we'll explore Python's principal built-in containers:- `list`- `set`- `tuple`- `dict`These can hold all sorts of objects, including instances of themselves!  ListsLists are a built-in Python data structure for grouping a collection of objects. You can create a list using square brackets `[ ]` or the built-in `list()` function<jupyter_code>a_list = [1, 'apple', True, None, 1.0] #see how a list can hold heterogeneous data
type(a_list)
another_list = list((2, 'banana', False, None, 2.0))
type(another_list)
2 in another_list #membership test using in<jupyter_output><empty_output><jupyter_text>Indexing and Slicing> You can index and slice lists the same way you did with strings.<jupyter_code>yet_another_list = ['a','b','c','d','e']

yet_another_list[0] #return the first element in the list
yet_another_list[::-1] #return the list in reverse order<jupyter_output><empty_output><jupyter_text>>But unlike strings, lists are **mutable**. This means they support item assignment. One way to do this is with indexing and slicing:<jupyter_code>yet_another_list[1:3] = ['x', 'y'] #replace the second and third elements with an x an y
yet_another_list<jupyter_output><empty_output><jupyter_text>> You can also remove elements from a list:<jupyter_code>yet_another_list[1:3] = []   #remove the elements by assigning an empty list
yet_another_list<jupyter_output><empty_output><jupyter_text>> But be careful if you're working with copies of a list. Sice lists are mutable, changing hte copy will also change the original.<jupyter_code>first_list = [1,2,3]
first_list_copy = first_list

first_list_copy[0] = 'a'
first_list<jupyter_output><empty_output><jupyter_text>List MethodsThere are also a lot of helpful list methods. You can find a full list [here](https://docs.python.org/3/tutorial/datastructures.htmlmore-on-lists), but these are two of the most useful ones:  `list.append()`> Add an item to the end of the list. Equivalent to `a[len(a):] = [x]`<jupyter_code>yet_another_list.append(2) #add 2 to the end of the list

yet_another_list #return the list to verify that the 2 has been added to the end<jupyter_output><empty_output><jupyter_text>`list.pop([i])`> Remove the item at the given position in the list, and return it. If no index is specified, `.pop()` removes and returns the last item in the list. (The square brackets `[]` around the `i` in the method denote that the parameter is optional, not that you should type square brackets at that position.)<jupyter_code>yet_another_list.pop() #take away and return that two we appended
yet_another_list #now yet_another_list no longer has the 2 that we popped<jupyter_output><empty_output><jupyter_text>List ConcatenationYou can also use the `+` operator to concatenate lists<jupyter_code>abc = ['a','b','c']
xyz = ['x','y','z']

abcxyz = abc + xyz
abcxyz<jupyter_output><empty_output><jupyter_text>Nested Lists> You can have lists inside of lists inside of lists inside of lists....> These are called nested lists and their elements can be accessed via indexing and slicing.<jupyter_code>nested_list = [['a','b','c'], [1,2,3], [1.0,2.0,3.0]]
nested_list #let's see what this looks like
nested_list[0]
nested_list[1]
nested_list[2][2]
nested_list[1:]<jupyter_output><empty_output><jupyter_text>Iterating Through Lists> Lists, like strings, are sequences. This means you can iterate through each element of a list. One of the easiest ways to do this is with a `for` loop.<jupyter_code>planets = ["Mercury", "Venus", "Earth", "Mars", "Jupiter", "Saturn", "Uranus", "Neptune", "AND PLUTO!"]
for planet in planets:
    print(planet)<jupyter_output>Mercury
Venus
Earth
Mars
Jupiter
Saturn
Uranus
Neptune
AND PLUTO!<jupyter_text>List Comprehensions> List comprehensions provide a concise way to create and iterate through lists. Common applications are to make new lists where each element is the result of some operation(s) applied to each member of another sequence, or to create a subsequence of those elements that satisfy a certain condition.> For example, say we wanted to create a list of the squares of the numbers 0-9. > We could use a `for` loop:<jupyter_code>squares = [] #first, we create an empty list that we'll fill during the for loop

for x in range(10):
    squares.append(x**2)
    
squares<jupyter_output><empty_output><jupyter_text>> Or we could use a `while` loop:<jupyter_code>squares = []
x = -1
while len(squares) < 10:  
    x += 1
    squares.append(x**2)

squares<jupyter_output><empty_output><jupyter_text>> The downside to those `for` and `while` loop methods, besides being verbose, is that they both create (or overwrite) a variable named `x`.> The much more elegant list comprehension won't have that [side effect](https://softwareengineering.stackexchange.com/questions/40297/what-is-a-side-effect):<jupyter_code>squares = [x**2 for x in range(10)]
squares<jupyter_output><empty_output><jupyter_text>> And here's an example of a list comprehension that creates a subsequence, taking the elements of squares that are divisible by 3.<jupyter_code>[x for x in squares if x%3 ==0]  #see how you can use conditional statements inside!<jupyter_output><empty_output><jupyter_text>> As you might have guessed, a list comprehension consists of brackets `[]` containing an expression followed by a `for` clause, then zero or more `for` or `if` clauses. The result will be a new list resulting from evaluating the expression in the context of the `for` and/or `if` clauses which followed it.> For example, this list comprehension combines the elements of two lists if they are not equal:<jupyter_code>combined = [[x, y] for x in [1,2,3] for y in [3,1,4] if x != y]
combined<jupyter_output><empty_output><jupyter_text>> The result is a nested list since our expression above `[x, y]` evaluates to a list. We could *flatten* this nested list with another list comprehension:<jupyter_code>[element for sublist in combined for element in sublist]<jupyter_output><empty_output><jupyter_text>> Of course, we could have used a `for` loop to flatten `combined`:<jupyter_code>flattened = []
for sublist in combined:
    for element in sublist:
        flattened.append(element)
        
flattened<jupyter_output><empty_output><jupyter_text>Nested List Comprehensions. > Consider the following matrix 3x4 matrix, which is a list of 3 lists of length 4:<jupyter_code>matrix = [
        [1, 2, 3, 4],
        [5, 6, 7, 8],
        [9, 10, 11, 12]
        ]
matrix<jupyter_output><empty_output><jupyter_text>> The following will transpose rows and columns:<jupyter_code>[[row[i] for row in matrix] for i in range(4)]<jupyter_output><empty_output><jupyter_text>> As you can imagine, nested list comprehensions - or their `for`/`while` loop equivalents - can quickly become hard to read. When possible, you should lean on built-in functions, like [`zip()`](https://docs.python.org/3.6/library/functions.htmlzip):<jupyter_code>list(zip(*matrix))<jupyter_output><empty_output><jupyter_text>Exercises > 1) Create an empty list `tenpeat` and fill it with the string "repeat" 10 times using a `for` loop that utilizes `range()`. > 2) Use a `for` loop with `range()`, where range accepts `len()` as the stop parameter, to `print()` each element of the list `planets`. > 3) Use a list comprehension to print each element from `planets`. > 4) Reverse the order of all of the elements in the list below.<jupyter_code>reverse_this_list = [0,1,2,3,4,5,6,7,8,9,10]<jupyter_output><empty_output><jupyter_text>> 5) Use a nested `for` loop to print each element from `nested_list` above. > 6) Use a nested list comprehension to print each element from `nested_list`.  SetsA `set` is an unordered collection with no duplicate elements. Basic uses include membership testing and eliminating duplicate entries. Set objects also support mathematical operations like union, intersection, difference, and symmetric difference.To create a set, you can use curly braces `{}` or the `set()` function. But to create an empty set, you have to use set(), not {}; the latter creates an empty dictionary, a data structure that we will discuss in the next section.<jupyter_code>grocery_list = ['apples', 'cookies', 'bananas', 'cookies', 'cookies', 'cookies', 'bread']
seta = set(grocery_list)
seta<jupyter_output><empty_output><jupyter_text>Above, we used `set()` to remove duplicates from `grocery_list` (obviously I wanted lots of cookies).  Common Set OperationsCommon set operations include union (`|`), intersection (`&`), difference (`-`), and symmetric difference (`^`). In addition to using those operators, there are also built-in set methods that do the same. To understand how these work, check out this visual:  `set.intersection()` corresponding operator:  `&`> Return a new set with elements that are common to all sets.> If we had other grocery lists and had to determine the set of items shared between the sets, we could use this method:<jupyter_code>setb = {'apples','bread','shampoo','ice cream'}
setb.intersection(seta)
seta & setb<jupyter_output><empty_output><jupyter_text>`set.difference()` corresponding operator:  `-`> The difference of sets A and B is a set of elements that exists only in set A but not in B<jupyter_code>seta.difference(setb)
seta - setb<jupyter_output><empty_output><jupyter_text>`set.union()` corresponding operator:  `|`> The union of sets A and B is a set with all of the elements from both sets.<jupyter_code>seta.union(setb)
seta | setb<jupyter_output><empty_output><jupyter_text>`set.symmetric_difference()` corresponding operator:  `^`> The symmetric difference of sets A and B is a set that has elements which are unique to each set.<jupyter_code>seta.symmetric_difference(setb)
seta ^ setb<jupyter_output><empty_output><jupyter_text>Sets and Venn Diagrams> Sets naturally lend themselves to venn diagrams in Python. Here's an example of a venn diagram. You might need to install the `matplotlib_venn` package. See [this](https://pypi.python.org/pypi/matplotlib-venn).<jupyter_code>from matplotlib import pyplot as plt
import numpy as np
from matplotlib_venn import venn3, venn3_circles
plt.figure(figsize=(4,4))
v = venn3(subsets=(1, 1, 1, 1, 1, 1, 1), set_labels = ('A', 'B', 'C'))
v.get_patch_by_id('100').set_alpha(1.0)
v.get_patch_by_id('100').set_color('white')
v.get_label_by_id('100').set_text('Unknown')
v.get_label_by_id('A').set_text('Set "A"')
c = venn3_circles(subsets=(1, 1, 1, 1, 1, 1, 1), linestyle='dashed')
c[0].set_lw(1.0)
c[0].set_ls('dotted')
plt.title("Sample Venn diagram")
plt.annotate('Unknown set', xy=v.get_label_by_id('100').get_position() - np.array([0, 0.05]), xytext=(-70,-70),
             ha='center', textcoords='offset points', bbox=dict(boxstyle='round,pad=0.5', fc='gray', alpha=0.1),
             arrowprops=dict(arrowstyle='->', connectionstyle='arc3,rad=0.5',color='gray'))
plt.show()<jupyter_output><empty_output><jupyter_text>Sets are mutable<jupyter_code>set_x = set([1, 2, 3])
set_y = set_x
set_y |= set([4, 5, 6])  # | is the bitwise-OR operation. So |= is to | as += is to +.
print(set_x)
print(set_y)<jupyter_output>{1, 2, 3, 4, 5, 6}
{1, 2, 3, 4, 5, 6}<jupyter_text>But you *can* make a set immutable by using [frozenset](https://docs.python.org/3.6/library/stdtypes.htmlfrozenset).  TuplesA tuple consists of a number of values separated by commas, for instance:<jupyter_code>t = 1, 2, 'meow!'
t<jupyter_output><empty_output><jupyter_text>On output tuples are always enclosed in parentheses, so that nested tuples are interpreted correctly.<jupyter_code>tup = t, ['list', 'inside', 'a', 'tuple'] #add a list to the tuple
tup<jupyter_output><empty_output><jupyter_text>Creating empty tuples and singletons> A special problem is the construction of tuples containing 0 or 1 items. There's special syntax for this. > Empty tuples are constructed by an empty pair of parentheses<jupyter_code>empty_tup = ()
empty_tup<jupyter_output><empty_output><jupyter_text>> A tuple with one item is constructed by following a value with a comma:<jupyter_code>singleton = 1,
singleton<jupyter_output><empty_output><jupyter_text>Tuples are immutable> Tuples may seem similar to lists, but they are often used for different purposes. That's because tuples are **immutable** and usually contain a heterogeneous sequence of elements. Since tuples are immutable, they are often manipulated more quickly than lists.<jupyter_code>tup[1] = 123 #tuples are immutable so you can't assign to them.<jupyter_output><empty_output><jupyter_text>> Although tuples are immutable, they can contain mutable objects just like lists. So if a value within a tuple is mutable, you can change it:<jupyter_code>t = ('a', 'b', ['c', 'd'])
t[2][0] = 'C'
t<jupyter_output><empty_output><jupyter_text>Tuple Unpacking> You can do the reverse of `t = 1, 2, 'meow!'` like so:<jupyter_code>t = 1, 2, 'meow!'
x, y, z = t #unpack the tuple and store each of its 3 values in 3 variables
print("x is {}, y is {}, and z is {}".format(x,y,z))<jupyter_output>x is 1, y is 2, and z is meow!<jupyter_text>Tuple Methods> There are just 2 tuple methods: `count()` and `index()`.<jupyter_code>tupple = (0,1,2,3,4,5,6,7,8,9)
tupple.count(1)
tupple.index(1)<jupyter_output><empty_output><jupyter_text>Other Tuple OperationsA lot of other built-in functions and operations can be applied to tuples:> len()<jupyter_code>len(tupple)<jupyter_output><empty_output><jupyter_text>> max(), min()<jupyter_code>max(tupple)
min(tupple)<jupyter_output><empty_output><jupyter_text>> Slicing<jupyter_code>tupple[:3]<jupyter_output><empty_output><jupyter_text>DictionariesAs the name implies, dictionaries map *keys* to *values* so that you can look up *values* with a *key*. Unlike sequences, which are indexed by a range of numbers, dictionaries are indexed by keys, which can be any immutable type, like tuples, strings, integers or floats. The values that map to these keys can be of any type, including another dictionary.Note that the `key : value` pairs of a dictionary are unordered. Note also that keys are unique within a dictionary.A pair of braces creates an empty dictionary: `{}`. Placing a comma-separated list of `key:value` pairs within the braces adds a `key:value` pair to the dictionary.<jupyter_code>drink_dict = {"water" : "yes", "gatorade" : "yes", "syrup" : "no" }<jupyter_output><empty_output><jupyter_text>The main dictionary operation is to extract a value given the key:<jupyter_code>drink_dict['water']<jupyter_output><empty_output><jupyter_text>`dict()` > The `dict()` constructor builds dictionaries directly from sequences of key-value pairs. Below, we pass a list of tuples to `dict()`:<jupyter_code>dict([("water" , "yes"),("gatorade" , "yes"), ("syrup" , "no")])<jupyter_output><empty_output><jupyter_text>> When the keys are short strings, it is sometimes easier to specify `key:value` pairs using keyword arguments:<jupyter_code>dict(water='yes',gatorade='yes',syrup='no')<jupyter_output><empty_output><jupyter_text>`d.keys())`> This will return a dictionary's keys in arbitrary order. You can use `list()` to make the output a list.<jupyter_code>list(drink_dict.keys())<jupyter_output><empty_output><jupyter_text>> If you want the keys sorted, just use `sorted(d.keys())` instead:<jupyter_code>sorted(drink_dict.keys())<jupyter_output><empty_output><jupyter_text>Key Membership> To check whether a single key is in the dictionary, use the `in` keyword:<jupyter_code>'gatorade' in drink_dict<jupyter_output><empty_output><jupyter_text>Dictionary Comprehension> dict comprehensions can be used to create dictionaries from arbitrary key and value expressions:<jupyter_code>{animal: animal+'-dog' for animal in ('cat', 'bird', 'lizard')}<jupyter_output><empty_output><jupyter_text>Looping through dictionaries> When looping through a dictionary, the keys are accessed by default:<jupyter_code>for key in drink_dict:
    print(key)<jupyter_output>water
gatorade
syrup<jupyter_text>> The key and corresponding value can be retrieved at the same time using the `items()` method:<jupyter_code>for key, value in drink_dict.items():
    print(key, value)<jupyter_output>water yes
gatorade yes
syrup no<jupyter_text>`enumerate()`> When looping through a dictionary - or any sequence for that matter - the index and corresponding value can be retrieved at the same time using the `enumerate()` function.<jupyter_code>for index, key in enumerate(drink_dict):
    print(index, key)<jupyter_output>0 water
1 gatorade
2 syrup<jupyter_text>FunctionsYou've already seen many examples of function calls. Here's an example:<jupyter_code>len('123')<jupyter_output><empty_output><jupyter_text>Above, the function *name* is `len` and it accepted one *argument*: `'123'`.When you *executed* or *called* the function, it returned the length of its argument:  3. This is often called the *return value*.So far, you've been using either Python's built-in functions - like `len()` - or functions imported from modules - like `math.sqrt()`. But you can also *define* your own functions. This allows you to utilize code in more than one place.  Defining a Function> How to define a function:  Declare the function using the keyword `def` followed by the function name.   - [Function names](https://www.python.org/dev/peps/pep-0008/function-names) should be lowercase, with words separated by underscores as necessary to improve readability  Write the arguments inside the opening and closing parentheses of the function, and end the `def` declaration with a colon `:`.  Write a [docstring](https://www.python.org/dev/peps/pep-0257/) using triple quotes (not mandatory, but helpful and good practice)  Write program statements  End the function with `return` a statement (there are other statements that will exit the function as well, but we'll leave that to you and Google to figure out)> Here's an example:<jupyter_code>def incrementer(number):
    """Increment a number by 1
    
    Arguments:
    number -- any numeric type, such as int, float, bool or complex
    """
    
    n = number
    result = n + 1
    return result
incrementer(1) # let's call the function by passing it the value 1<jupyter_output><empty_output><jupyter_text>> Above, we defined a function named `incrementer` that accepts one **argument**:  `number`. Then we ended the `def` declaration with a colon `:`.> Next we wrote our **docstring**. The first line of the docstring explains what this function does. Then we describe the function's one argument by noting the types we intended for it.> Then we created a **parameter** called `n` that takes the value of `number` from the function's argument. This step was optional.> Then we created a **variable** called `result` and assigned to it the value of `n + 1`.> Finally, there's a `return` statement, which returns the value of `result`.  Arguments> When defining a function, you can define your argument(s) either implicitly (by position) or explicitly (by name).>  Positional Arguments> A positional argument is a name that is not followed by an equal sign (=) and default value. When you call a function with positional arguments, the order of the arguments you pass into the function call matters.<jupyter_code>def print_position(first, second):  #two positional arguments
    print(first)
    print(second)
print_position('first','second')
print_position('second','first')<jupyter_output>second
first<jupyter_text>>  Keyword Arguments>A keyword argument is followed by an equal sign and an expression that gives its default value. When you call a function with keyword arguments, the default value will be used if you don't specify the keyword argument in the function call.<jupyter_code>def print_keyword(first = 'first'): #with default values
    print(first)
print_keyword(first = 'second?!') #pass a value into the keyword argument
print_keyword() #or rely on the default value and don't pass anything into the function call<jupyter_output>first<jupyter_text>>  Positional Arguments with Keyword Arguments> If you want to use positional arguments with keyword arguments, you must define the positional arguments before the keyword arguments, otherwise you'll get a `SyntaxError`.<jupyter_code>def print_both_args(first, second = "second"):
    print(first)
    print(second)
print_both_args("first")
# this won't work
def print_both_args2(first = "first", second):
    print(first)
    print(second)<jupyter_output><empty_output><jupyter_text>> But you can pass a positional argument after a keyword argument if you treat it as a keyword argument:<jupyter_code>print_both_args(second = "second", first = "first")<jupyter_output>first
second<jupyter_text>>  Argument Collections> Functions can also accept two types of argument collections.> The first is a named `tuple` that contains any additional positional arguments. This special argument is defined by using a single asterisk (*) as a prefix.> The second is a `dictionary` containing any additional keyword arguments. This special argument is defined with two asterisks (**).> The convetion is to name these `args` and `kwargs`, respectively.<jupyter_code>def arg_collection(*args, **kwargs):
    print('args:', args)
    print('kwargs:', kwargs)
arg_collection('arg')
arg_collection(foo='kwargs')
def all_together(a, b, c=None, *args, **kwargs):
    print('(a, b, c):', (a, b, c))
    print('args:', args)
    print('kwargs:', kwargs)
all_together('a', 'b', 'c', 'd', 'e', f='f', g='g')<jupyter_output>(a, b, c): ('a', 'b', 'c')
args: ('d', 'e')
kwargs: {'f': 'f', 'g': 'g'}<jupyter_text>>  Unpacking Argument CollectionsYou can also create a list of arguments and pass them into a function.> For positional arguments, you insert them into a `tuple` or `list` and then prepend that object with an asterisk (`*`) in the function call.<jupyter_code>def print_stuff(a, b, c=0, d=None):
    print('a: {0}, b: {1}, c: {2}, d: {3}'.format(a, b, c, d))
positional_args = (1, 2)  #using a tuple
keyword_args = {'c': 3, 'd': 4} #using a dictionary
print_stuff(*positional_args, **keyword_args)<jupyter_output>a: 1, b: 2, c: 3, d: 4<jupyter_text>> Note that the function will still print even if we didn't use the asterisks. This can produce some unintended consequences. That's why `docstrings` are super helpful!<jupyter_code>print_stuff(positional_args,keyword_args)<jupyter_output>a: (1, 2), b: {'c': 3, 'd': 4}, c: 0, d: None<jupyter_text>Docstrings> A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition. A docstring becomes the `__doc__` special attribute of that object.<jupyter_code>print(incrementer.__doc__)        #print our function's docstring
help(incrementer) # you can also use the built-in help function<jupyter_output>Help on function incrementer in module __main__:

incrementer(number)
    Increment a number by 1
    
    Arguments:
    number -- any numeric type, such as int, float, bool or complex<jupyter_text>> You can also access docstrings by typing shift + tab while your cursor is within the parentheses of a function. Try that below:<jupyter_code>np.array()<jupyter_output><empty_output><jupyter_text>Value-Returning vs. Void Functions> In Python, you can define a function without a return statement. These are called **void functions** because they return `None`, a special Python type that represents "nothing". Let's see this in action:<jupyter_code>def void_function():
    print("Howdy! I'm a void function!")

def value_function():
    return "Howdy! I'm a value-returning function!"<jupyter_output><empty_output><jupyter_text>> Let's see what happens when we call each function:<jupyter_code>void_function()
value_function()<jupyter_output><empty_output><jupyter_text>> Note the subtle difference: `void_function()` printed our message while `value_function()` returned a `str` object (as indicated by the quotation marks). > To verify that `void_function()` returns `None` and that `value_function()` returns our message, let's try assigning these functions to some variables<jupyter_code>void = void_function() #void_function will still print
void == None #but the variable void holds None instead of the message
value = value_function()
value #the variable value holds the str object returned by value_function()<jupyter_output><empty_output><jupyter_text>Generalization> You can make a function more versatile by adding additional arguments. Let's generalize `incrementer` by adding an argument that lets us determine how much we want to increment.<jupyter_code>def incrementer(number, increment):
    """Increment a number by an increment, positive or negative
    
    Arguments:
    number -- any numeric type, such as int, float, bool or complex
    
    increment -- any numeric type, such as int, float, bool or complex
    """
    
    n = number
    result = n + increment
    return result
incrementer(1.0,-1)<jupyter_output><empty_output><jupyter_text>Parameters and Variables are "Local" > Parameters and variables only exist within their functions. In Python, this is the concept of **namespace** and **scope**, two subjects we'll treat in a [later lesson](https://github.com/csmcallister/learning-data-science/blob/master/Learn%20Python/3.%20Data%20Structures/5.%20Classes/Classes.ipynb).> For now, note that we created parameter `n` and variable `result` within the scope of our `incrementer` function. > Let's execute `incrementer` again to see how `n` and `result` are destroyed when the function terminates.<jupyter_code>incrementer(1,1)
n
result<jupyter_output><empty_output><jupyter_text>Enclosing Functions> Python allows you to nest functions so that you can pass variables from one function to another. Also called a **closure**, this allows an inner function to have access to the outer (enclosing) function's variables.> In the example below, `outer_function` passes off `a` to its enclosed `inner_function`:<jupyter_code>def outer_function():
    a=1
    def inner_function():
        print(a)
    inner_function()
outer_function()        #when you call outer_function, 
                        #note how the print function of inner_function is actually executed<jupyter_output>1<jupyter_text>> So why are enclosing functions important? >Consider the case when `a=1` is a computationally expensive operation that you'd like `inner_function` to have access to. Consider also that, instead of one `inner_function`, you have many. Instead of having to repeatedly type this verbose operation in each separate `inner_function` - and then having it computed on every function call - you could wrap the operation in a **closure** and have it precomputed for all of your `inner_functions`!  Refactoring> Code [refactoring](https://en.wikipedia.org/wiki/Code_refactoring) is the process of restructuring computer code to improve code readability without changing external behavior. Consider a simple program that needs to be able to calculate the following: - The mean of a list of $n$ numbers$${\bar{x} = \frac {1}{n}}\sum _{i=1}^{n}x_{i}$$ - The sum of the square root of the squared deviations from the mean of each value within a list of numbers$$\sum _{i=1}^{n}\sqrt{(x_i - \bar{x})^2}$$<jupyter_code>def mean(num_list):
    return sum(num_list) / len(num_list)

def sum_deviations(num_list):
    deviations = []
    num_list_mean = sum(num_list) / len(num_list)                #line 6
    for n in num_list:
        deviations.append(((n - num_list_mean)**2)**(1/2))       #line 8
    return sum(deviations)<jupyter_output><empty_output><jupyter_text>> Some of the code above is repetitive. Check lines 6 and 8, where we calculate the mean inside the `deviations` function. Since we already wrote code to calculate the mean of a list of numbers in the `mean` function, we could simplify `sum_deviations` by making calls to the `mean` function. Here's the refactoring:<jupyter_code>def mean(num_list):
    return sum(num_list) / len(num_list)

def sum_deviations(num_list):
    deviations = []
    #num_list_mean = sum(num_list) / len(num_list) ## <-- this whole line of code can be nixed
    for n in num_list:
        deviations.append(((n - mean(num_list))**2)**(1/2)) ## <-- and here's the call to the mean function
    return sum(deviations)<jupyter_output><empty_output><jupyter_text>> Of course this is a simplified example, but such refactoring can make a huge difference in code readability once you begin writing larger programs.  Recursion> Recursion is the process of defining something in terms of itself. Consider this form of visual recursion, called the [Droste Effect](https://en.wikipedia.org/wiki/Droste_effect):The woman is holding an object that contains a smaller image of her holding an identical object, which in turn contains a smaller image of herself holding an identical object, and so forth.> In computer science, recursion is a method where the solution to a problem depends on solutions to smaller instances of the same problem. > A **recursive function** is a function that calls itself over and over until some sort of condition - called a **base case** - is met. > Consider a [factorial](https://en.wikipedia.org/wiki/Factorial). In mathematics, the factorial of a non-negative integer $n$, denoted by $n!$, is the product of all positive integers less than or equal to $n$. For example:> $$5!=5\times 4\times 3\times 2\times 1=120$$> This is recursive because you're multiplying a number by itself minus one over and over again until you reach the base case of zero.> The recursive definition of a factorial is then the equations: $$0! = 1$$  $$n > 0, n! = n(n − 1)!$$Here it is in Python:<jupyter_code>def factorial(n):
    if n < 1:         #set a condition that will catch the base case before you hit Python's recursion limit
        return 1      #return 1 since 0! = 1
    else:
        answer = n * factorial(n - 1) #create a variable called answer that is the product of n and the recursive call
        #print(str(n)+'! = ',answer)   #this print statement can help you understand what's going on
    return answer
factorial(5)<jupyter_output><empty_output><jupyter_text>> Of course, you could also calculate a factorial without recursion using iteration:<jupyter_code>def iter_factorial(n):
    a = 1
    for i in range(2,n+1):
        a *= i
    return a

iter_factorial(5)<jupyter_output><empty_output><jupyter_text>Advantages of Recursion> - Since recursion often gels with mathematics, recursive functions can make code more elegant.- Recursion can also make it easier to break down a complex task into simpler sub-tasks. Disadvantages of Recursion> - Recursive logic can quickly become tough to folow; this can make recursive functions tough to debug- Recursive calls are inefficient as they take up a lot of memory and time (play with the function below to see this in action)<jupyter_code>import timeit
def time_tester(i):
    """Print how long recursion and iteration took for calculating a factorial.
    
    Arguments
    i - an integer for calculating i! (don't make this number too big!)
    """
    r_time = timeit.timeit('factorial('+str(i)+')', globals=globals())
    i_time = timeit.timeit('iter_factorial('+str(i)+')', globals=globals())
    print("Recursion for {}! took approximately {:.2f} seconds".format(i, r_time))
    print("Iteration for {}! took approximately {:.2f} seconds".format(i,i_time,))
    print("Iteration was {:.1f} times faster than recursion".format(r_time/i_time))
time_tester(10)  #enter an integer to see the time difference<jupyter_output>Recursion for 10! took approximately 2.72 seconds
Iteration for 10! took approximately 1.20 seconds
Iteration was 2.3 times faster than recursion<jupyter_text>Exercises > 1) Write a function that squares each value in a list, returning the result as a new list. > 2) Write a function that returns the measure of the interior angle of an n-sided polygon. Here's the formula for calculating the measure of an n-sided polygon's interior angle:$$\frac{(n-2) \times 180}{n}$$ > 3) Write a function that tests if a number is [prime](https://en.wikipedia.org/wiki/Prime_number). Note that you can test if a number is prime by seeing if it is divisible by any number up to its square root. > 4) A [palindrome](https://en.wikipedia.org/wiki/Palindrome) is a word or phrase that reads the same backward as forward, such as madam, “taco cat”, or racecar. Write a function that tests if its argument, which should be a string, is a palindrome. Hint: this can be solved with or without recursion.  Errors and Exceptions Until now, we've encountered error messages without really inspecting them. There are (at least) two distinguishable kinds of errors: **syntax errors** and **exceptions**.  Syntax Errors> Also known as parsing errors, syntax errors are the most common complaint you'll get while learning Python:<jupyter_code>print('Hello world'<jupyter_output><empty_output><jupyter_text>> When there's a syntax error, the parser repeats the offending line and displays a little ‘arrow’ pointing at the earliest point in the line where the error was detected. The error is caused by (or at least detected at) the token preceding the arrow. > In the example above, the error is detected at the end of the function `print()` since it's missing a closing parenthesis `)`. The file name and line number are printed so you know where to look in case the input came from a script.  Exceptions> Even if your code is syntactically correct, it may cause an error upon execution. Errors detected during execution are called **exceptions**. Unlike syntax errors, exceptions are not unconditionally fatal. This means you can write code to handle them. If you don't handle them, then you'll be stuck with errors. Consider the following three errors:<jupyter_code>1 / 0
foo*2
'1' + 1<jupyter_output><empty_output><jupyter_text>> Each of the above demonstrates a [built-in exception](https://docs.python.org/3/library/exceptions.htmlbltin-exceptions), although it is possible to create your own user-defined exceptions.> Exceptions come in different types, and the type is printed as a part of that message. Above we see `ZeroDivisionError`, `NameError` and `TypeError` types. The rest of the message provides details on the type of exception and what caused it.  Handling Exceptions> It is possible to write programs that handle selected exceptions. Look at the following example, which asks the user for input until a valid integer has been entered:<jupyter_code>while True:
    try:
        x = int(input("Please enter a number: "))
        break
    except ValueError:
        print("Oops!  That was no valid number.  Try again...")<jupyter_output>Please enter a number: a
Oops!  That was no valid number.  Try again...
Please enter a number: b
Oops!  That was no valid number.  Try again...
Please enter a number: c
Oops!  That was no valid number.  Try again...
Please enter a number: 1<jupyter_text>> Here's how the `try` statement works:- First, the `try` clause (the statements between the `try` and `except` keywords) is executed.- If no exception occurs, the `except` clause is skipped and execution of the `try` statement is finished.- If an exception occurs during execution of the `try` clause, the rest of the clause is skipped. Then if its type matches the exception named after the `except` keyword, the `except` clause is executed, and then execution continues after the `try` statement.- If an exception occurs which does not match the exception named in the `except` clause, it is passed on to outer `try` statements; if no handler is found, it is an unhandled exception and execution stops with an error message.  Why Handle Exceptions?> In Python, you're meant to use exceptions for control flow. For example, the iterator protocol, which is how `for` loops work, handles a `StopIteration` error to signal loop termination. Since exception handling is a part of the language, using it can help you avoid unforseen problems. > Consider the task of reading a data file without knowing whether the file is going to be JSON or YAML:<jupyter_code>import json
 
import yaml


def parse_file(filename):
    try:
        return json.load(open(filename))
    except json.JSONDecodeError:
        return yaml.load(open(filename))
parse_file("assets/yavin4.json") #parse some json
parse_file("assets/invoice.yaml") #parse some yaml
parse_file("this file doesn't exist.yaml")<jupyter_output><empty_output><jupyter_text>> Our function was able to parse both JSON and YAML thanks to exception handling. But it still let us know when it encountered another exception (i.e. `FileNotFoundError`). This is good in case you had wanted to try YAML parsing if and only if JSON parsing had failed due to `JSONDecodeError`.> There's a lot more to exception handling. You can read more about it [here](https://docs.python.org/3/tutorial/errors.html).  What's Next > Next up we'll treat a specialized data structure: the `array`. There's a module called `numpy`, which is the core library for scientific computing in Python, that allows us to create and work with arrays. After arrays, we'll be ready for the `pandas` module, which will allow us to work with multidimensional, structured data sets (aka **pan**el **da**ta **s**tructures).  The Star Wars API:  Parsing JSON Using Lists & DictionariesThe [Star Wars API](https://swapi.co/) is the world's first quantified and programmatically-formatted set of Star Wars data. With Python, we can make *requests* from this dataset and then work with it.  JSON> JavaScript Object Notation, or [JSON](https://en.wikipedia.org/wiki/JSON), is an open-standard file format for transmitting data on the web. Popular alternatives to JSON are [YAML](https://en.wikipedia.org/wiki/YAML) and [XML](https://en.wikipedia.org/wiki/XML).> Like many other APIs, The Star Wars API uses JSON. Fortunately for us, JSON is remarkably similiar to a Python dictionary and the `json` module provides a quick and easy way to convert JSON to a dictionary.<jupyter_code>import requests, json                           #the requests module is used to get the data from the API
r = requests.get('http://swapi.co/api/films/1') #get the json from the API
a_new_hope = r.json()                           #convert the json to a dictionary
a_new_hope.keys()                               #return all of the keys from the dictionary<jupyter_output><empty_output><jupyter_text>> If we pass the `'opening_crawl'` key to the dictionary, we can access the value:<jupyter_code>print(a_new_hope['opening_crawl'])<jupyter_output>It is a period of civil war.
Rebel spaceships, striking
from a hidden base, have won
their first victory against
the evil Galactic Empire.

During the battle, Rebel
spies managed to steal secret
plans to the Empire's
ultimate weapon, the DEATH
STAR, an armored space
station with enough power
to destroy an entire planet.

Pursued by the Empire's
sinister agents, <NAME> races home aboard her
starship, custodian of the
stolen plans that can save her
people and restore
freedom to the galaxy....<jupyter_text>> Let's check out the characters in the movie:<jupyter_code>a_new_hope['characters']<jupyter_output><empty_output><jupyter_text>> The value corresponding to the `characters` key in `a_new_hope` is a `list` of urls. We can use a loop to:1. pass each of these urls in another `requests.get()` call2. convert each JSON response to a dictionary3. append each dictionary to a list<jupyter_code>char_list = [] #instantiate an empty list

for url in a_new_hope['characters']:
    r = requests.get(url) #pass each url into a requests.get() call
    char_list.append(r.json()) #convert each JSON response to a dictionary and append it to the list

char_list[0] #inspect the first character in our list of dictionaries<jupyter_output><empty_output><jupyter_text>> Looks like we found Luke. What if we wanted to get a list of all the male characters from A New Hope?<jupyter_code>[x['name'] for x in char_list if x['gender'] == 'male']<jupyter_output><empty_output><jupyter_text>> Now what if we wanted to plot the mass and height of each male character from A New Hope?<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

data = [(x['name'],x['mass'],x['height']) for x in char_list if x['gender'] == 'male'] #create list of tuples for chars
df = pd.DataFrame(data) #convert data to dataframe object
df.columns = ['Name','Mass','Height'] #rename columns
df['Height'] = pd.to_numeric(df['Height'], errors = 'coerce') #set datatype
df['Mass'] = pd.to_numeric(df['Mass'], errors = 'coerce') #set datatype

sns.lmplot("Height", "Mass", data=df, hue='Name', fit_reg=False) #plot Mass against Height<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09770412973788421, 'fraction_numerical': 0.008226475579860457, 'mean_word_length': 5.229809104258443, 'pattern_counts': {'":': 0, '<': 325, '<?xml version=': 0, '>': 451, 'https://': 19, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '84', 'dir': 'jupyter-structured-clean-dedup', 'id': '483599', 'license': 'Unlicense', 'n_tokens_mistral': 12984, 'n_tokens_neox': 12351, 'n_words': 5213, 'path': 'Learn Python/3. Data Structures/1. Containers and Control Flow/Containers and Control Flow.ipynb', 'repo_name': 'mcallistercs/learning-data-science', 'size': '42432'}