<jupyter_start><jupyter_text>Lecture 2: Intro to Python 2/20/19 Table of Contents0. [Why Python?](why-python)1. [Basic Python Syntax](basic-syntax)      1.1 [Jupyter](jupyter)      1.2 [Code](code)      1.3 [Comments](comments)      1.4 [Errors](errors)      1.5 [Numbers and Arithmetic](numbers)  2. [Variables and Functions](vars-functions)      2.1 [Variables](vars)      2.2 [Control Flow](control-flow)      2.3 [Functions](funcs)  3. [Arrays and Iteration](arrs-iter)      3.1 [Arrays and Lists](arrs)      3.2 [Dictionaries](dicts)      3.3 [For Loops](for)  4. [Exercises](exercises)5. [Homework](homework)6. [Resources](resources) Hosted by and maintained by the [Statistics Undergraduate Students Association (SUSA)](https://susa.berkeley.edu). Authored by [<NAME>](mailto:<EMAIL>) and [<NAME>](mailto:<EMAIL>)  Why Python? Python is a widely used programming language, and it's one of the easiest to learn because of how similar it is to [psuedocode](https://en.wikipedia.org/wiki/Pseudocode) - basically, it looks really similar to regular English, so it can be easier to pick up and understand for people who are new to programming!That doesn't mean you should underestimate the power of Python! Despite being "simple", it's used for a wide array of computational tasks, including backend web development, image processing, and data analysis (which is what we will focus on this semester).Because of how popular it is, there's also a large community of Python developers that work on 3rd-party Python packages to simplify a lot of tasks. Many of these are open-source projects, which means you can look at the code directly and even contribute to it (if you want). This also means that a lot of the more popular packages have been thoroughly tested for bugs and fixed by thousands (millions?) of people. You can browse some of these packages at the [Python Packge Index (PyPI)](https://pypi.org), and we'll be using a lot of these packages throughout this semester.  1. Basic Python Syntax  1.1 Jupyter First things first! We're gonna learn how to navigate this screen you're looking at right now. This is called a [Jupyter Notebook](https://jupyter.org). It's an interactive notebook that can both render text and run code! **Fun fact!** The creator of the Jupyter Notebooks (formerly known as the IPython Notebook), <NAME>, is a UC Berkeley Department of Statistics professor and has attended several of SUSA's faculty dinners! If you click anywhere on the notebook, you'll notice the content is highlighted with a box around it. Each of these boxes is called a *cell*. In Jupyter notebooks, there are two main types of cells: A. *Markdown (Text)*- These are the ones you've seen so far- You can edit by double-clicking or pressing `enter` while highlighted- They just consist of text and you can do some cool things like making this list or *italics* or **bold**- If you want to read more about what you can do with Markdown check out the link [here](github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) **Practice**: Edit the cell below with your *deepest darkest secret* I hate chocolate B. *Code*- cells that contain code (like Python)- running a cell will execute all of its contents- can be edited by single-clicking on its contents or by pressing `enter` while highlighted- to run the code in a cell press the `▶| Run` button or press `shift` + `enter` while highlighted **Practice**: Try running the cell below<jupyter_code>print("hello world")<jupyter_output><empty_output><jupyter_text>Jupyter also has some handy keyboard shortcuts. You can see all of them by clicking the `Help` toolbar and `Keyboard Shortcuts`, but here's a few you'll probably be using a lot:- `esc`: allows you to exit editing mode and manipulate cells and navigate using arrow keys- `esc` + `h`: open keyboard shortcuts- `esc` + `p`: open command palette (search through commands and shortcuts)- `esc` + `m`: change cell to a markdown cell- `esc` + `y`: change cell to a code cell- `esc` + `a`: insert a new code cell above- `esc` + `b`: insert a new code cell below- `esc` + `d` + `d`: delete the current cell- `shift` + `enter`: run current cell and move cursor to next cell- `ctrl` + `enter`: run current cell and don't move cursor- `opt/alt` + `enter`: run current cell and create new cell belowTake  moment to try out each of these shortcuts so you get a feel for what they do!  1.2 Code In Python, each command is written on a separate line, and each line is executed in order.<jupyter_code>print("hello")
print("world")<jupyter_output><empty_output><jupyter_text>**Practice**: Alter the code above so that the output looks like this:    rosa is my favorite s u s a member    today    is    patrick's    birthday  1.3 Comments Sometimes, you might see something that looks like this:<jupyter_code># This is a comment
# Any line that starts with `#` is ignored by the computer when executing code
"""
Comments can also look like this (this is called a docstring)
These are multi-line comments that are often used to describe what functions do
""" 
print("Python will still execute non-commented lines in the same cell!")
# print("but this will be ignored")
# pls notice me :'(<jupyter_output><empty_output><jupyter_text>1.4 Errors Everyone makes mistakes, everyone has those days, everyone knows what I'm talking about...luckily Python has come to save the day so you *know* when something you've written is syntactically wrong! Python informs you of mistakes through *error messages*. Try running the cell below to see what an error message looks like:<jupyter_code>print("hello"<jupyter_output><empty_output><jupyter_text>The error message tells you where your error is and what went wrong. In this case we made a `SyntaxError` , which means the structure of our code is flawed. There are many different types of `Error`s that you'll encounter in your Python adventures, but `SyntaxError`s are probably some of the most common you'll see.**Practice**: Try to fix the `SyntaxError` in the cell above  1.5 Numbers and Arithmetic One of the most basic things you can do with Python is math! You can use it as a calculator to perform simple functions like addition, subtraction, multiplication, etc, etc. It'll take any numerical `expression`, which can be a single number or a combination of numbers and operators/functions and evaluate them line by line.<jupyter_code>12.100000<jupyter_output><empty_output><jupyter_text>If you ran the cell above, you'll notice that it outputted `12.1` even though you didn't explicitly `print()` it. Whenever you run a code chunk, Jupyter will output the value of the very last line evaluated, if it has one. However, it will only do this for the very last line!What do you expect will happen when you run the cell below?<jupyter_code>print(2)
1
2
3<jupyter_output><empty_output><jupyter_text>You can also do basic arithmetic functions like: Addition: `+`<jupyter_code>2 + 3<jupyter_output><empty_output><jupyter_text>Subtraction: `-`<jupyter_code>20 - 7<jupyter_output><empty_output><jupyter_text>Multiplication: `*`<jupyter_code>3 * 5<jupyter_output><empty_output><jupyter_text>Division: `/`<jupyter_code>5 / 3<jupyter_output><empty_output><jupyter_text>Floor Division: `//`<jupyter_code>5 // 3<jupyter_output><empty_output><jupyter_text>Modulo (Remainder): `%`<jupyter_code>5 % 3<jupyter_output><empty_output><jupyter_text>Exponentiation: `**`<jupyter_code>3 ** 2<jupyter_output><empty_output><jupyter_text>Make sure there are no spaces between the `*`! Otherwise there will be an error Python follows the order of operations!<jupyter_code>12 + 4 * 7 - 6 * 3 ** 4 * 2 ** 5 / 3 * 6<jupyter_output><empty_output><jupyter_text>2. Variables and Functions Stringing together arithmetic functions like above is all fine and dandy, but it can be hard to read. What if you want to reuse the same value that you've calculated? Should you just copy and paste the expression every time? What if you wanted to calculate the same value as above, but with different numbers? Should you copy and paste and change the numbers manually? Nope! A better (more readable, more efficient) way to write code is through using *variables* and *functions*.  2.1 Variables Variables allow you to save values to named objects by using *assignment statements*. In Python an assignment statement is of the form    {variable name} = {value}Variable names can be any combination of letters, numbers, and underscores (`_`), starting with a letter or underscore. A variable name starting with an underscore is generally for internal use, so for now, don't use them.<jupyter_code>x = 3
y = "potato"<jupyter_output><empty_output><jupyter_text>Are these valid variable names?- susa2019- 2019susa- susa_is_great- susa-is-great- susaIsGreat- susaIsGreat!- \_susa All variables must be defined before they're used, otherwise you'll get an error, because Python doesn't know what you're talking about.<jupyter_code>aylmao<jupyter_output><empty_output><jupyter_text>Each variable has a different `type`. The type of a variable determines what you can do with it, following Python's type rules. Let's check what types our variables `x` and `y` are. You can do this by using the `type()` function.<jupyter_code>print() # print the type of x

print() # print the type of y<jupyter_output><empty_output><jupyter_text>**Question**: Logically, what do you think should happen when you run the cell below?<jupyter_code>x + y<jupyter_output><empty_output><jupyter_text>As you can see above, when you try to perform operations on objects that are the wrong type, you get a `TypeError`. There are several built-in types in Python, including the ones you've seen already. We'll look at some basic ones here: A. *Strings* (`str`)- any combination of characters surrounded by double-(") or single-(') quotes<jupyter_code>string1 = "potato"
string2 = 'what'<jupyter_output><empty_output><jupyter_text>B. *Numerics*- envelopes many subtypes that all represent numbers- Two common types:     - `int`: represent integers (whole numbers)    - `float`: represent numbers with decimals<jupyter_code>int1 = 5
float1 = 22.3<jupyter_output><empty_output><jupyter_text>C. Booleans (`bool`)- represent truth values- only two possible values:    - `True`    - `False`<jupyter_code>bool1 = True
bool2 = False<jupyter_output><empty_output><jupyter_text>For working with `bool`s, we will introduce three more operators: 1. `and`    - returns `True` if the expressions on both sides evaluate to `True`<jupyter_code>bool1 and bool2<jupyter_output><empty_output><jupyter_text>2. `or`    - returns `True` if either of the expressions evaluate to `True`<jupyter_code>bool1 or bool2<jupyter_output><empty_output><jupyter_text>3. `not`    - returns the opposite boolean value of the expression on the right<jupyter_code>not bool1<jupyter_output><empty_output><jupyter_text>The primitive operators we've talked about already (`+`, `-`, `*`, `/`) work with some of these built-in types. Try different combinations of the variables above to see which operators work on which variables!<jupyter_code>string1 + string2
string1 * int1<jupyter_output><empty_output><jupyter_text>2.2 Control Flow We can use `bool`s to control how our programs execute. *Control flow* allows us to decide if we want to conditionally execute code, or run a certain line of code multiple times. Here are a couple control flow operations. A. `if`- allows you to execute a chunk of code only if a certain condition is satisfied<jupyter_code>if int1 < 10:
    int1 = int1 + float1
    
int1<jupyter_output><empty_output><jupyter_text>We can make these if statements more complex by adding `elif` and  `else`.<jupyter_code>if int1 < 10:
    int1 = int1 + float1
elif int1 > 10:
    int1 = int1 - float1
else:
    print("omg int1 is 10!")<jupyter_output><empty_output><jupyter_text>The `elif` says "if the previous statements aren't true but this one is true, execute the code in this body". The `else` says "if all previous statements weren't true, unconditionally execute this body". You can have as many `elif` statements as you want, but you can only have one `if` and `else`, which must be placed at the top and bottom of your if-elif-else chain, respectively. B. `while`- allows you to continuously run the same chunk of code while a certain condition is met<jupyter_code>while int1 < 10:
    int1 = int1 + 1
    print(string1)
    
int1<jupyter_output><empty_output><jupyter_text>**Practice**: Try printing your name as many times as your age, i.e. if you're 20, print your name 20 times, without using 20 print statements  2.3 Functions Another type of object in Python are *functions*. Functions allow you to give a name to a set of expressions, in the case that you want to be able to repeat an action on various inputs or reuse logic without copy and pasting. Functions can be defined in the following form:     def {function name}:        {function body}        The same rules for naming variables apply to naming functions. Here's an example of a simple function that we might want to use. This allows us to square any input without having to type out `x * x` every time we want to square something.<jupyter_code>def square(x):
    return x * x<jupyter_output><empty_output><jupyter_text>Now we can call this function on any valid inputs!<jupyter_code>print(square(2))
print(square(1.3))<jupyter_output><empty_output><jupyter_text>What do you think will happen if we try to call `square()` on a string?<jupyter_code>square('hello')<jupyter_output><empty_output><jupyter_text>This brings us back to our type definitions from before. Unlike some other programming languages, Python doesn't explicitly type variables, so we don't know what type of input `square()` takes unless we read through the code ourselves. This is why it can be helpful to write docstrings for your functions, so that someone who comes along later can read what inputs your function takes, what it does, and what it outputs.<jupyter_code>def square(x):
    """
    Returns x squared
    
    Parameters:
        x (numeric): number to be squared
    
    Returns:
        x squared
    
    """
    return x*x<jupyter_output><empty_output><jupyter_text>A helpful tool that Jupyter has is a built-in tooltip which will give you information about a function. You can access it while your cursor is on the name of a function by pressing `shift` + `tab`. This will tell you the name of the function and its parameters, as well as a brief description of what the function does. If you press `tab` a second time, the tooltip will expand with more information including argument and return types, as well as some examples. If you press `tab` a third time, the tooltip will stick to the bottom of the screen. Try it on the `print()` and `square()` functions below:<jupyter_code>print()<jupyter_output><empty_output><jupyter_text>**Practice**: Try writing a function that takes in two arguments and returns the first argument raised to the power of the 2nd argument.*Hint*: try using a loop<jupyter_code>def power(a, b):
    """
    Fill me in
    """
    ## Fill me in!<jupyter_output><empty_output><jupyter_text>3. Arrays and Iteration  3.1 Arrays and Lists Another major data type in Python is a `list`. Lists allows you to store an ordered collection of items that you can add to.<jupyter_code>x = [1, 1, "hi", True]
x<jupyter_output><empty_output><jupyter_text>Each item in the list has an `index`, which is basically just its position. You can access an item in the list by selecting its index. Python (as well as most programming languages you'll come across) is 0-indexed, which means indices start at 0. You can index into lists in the following way:    list[index] **Practice**: Try printing the 3rd element in the list `x`. You can also replace specific items to a list by assigning to an element by index. Try changing the first value to `2`. Python has a built-in function called `len()`. Use the tooltip to figure out what this function does and try printing the last element in the list `x`.<jupyter_code>len<jupyter_output><empty_output><jupyter_text>Some objects in Python have *methods*, which are functions that are defined for that specific type. You can invoke a method on a variable by calling it in the following format:    {variable name}.{method name}    This is equivalent to calling    {object type}.{method name}({variable name}) Once again, Jupyter has implemented a helpful tool for searching through all of an object's methods. If you type in `{variable name}.` and then press `tab` after the `.`, a scrollable list of valid methods will pop up.<jupyter_code>x.<jupyter_output><empty_output><jupyter_text>You can try playing with the functions to figure out what they do, but here are some of the main ones for reference. A. `append(item)`   - adds `item` to the end of the list<jupyter_code>x.append(4)
x<jupyter_output><empty_output><jupyter_text>B. `pop(item)`- removes and returns the last item in the list<jupyter_code>x.pop()<jupyter_output><empty_output><jupyter_text>C. `insert(pos, item)`- adds an item at position `pos`, pushing all other elements to the right<jupyter_code>x.insert(2, 'happy')
x<jupyter_output><empty_output><jupyter_text>D. `remove(item)`- removes the element with value `item`, shifting all elements on the right to the left<jupyter_code>x.remove('happy')
x<jupyter_output><empty_output><jupyter_text>You can also check if a value is in a list using the following syntax:    {value} in {list}<jupyter_code>'happy' in x<jupyter_output><empty_output><jupyter_text>A list can contain values of any type, including other lists! When you have a list that has other lists as elements, it's called a *nested list*.<jupyter_code>nested = [[1, 2, 3], ['a', 'b', 'c'], [True, False]]<jupyter_output><empty_output><jupyter_text>**Practice**: Can you think of how you'd be able to get the value `'b'` from the nested list above?<jupyter_code>nested<jupyter_output><empty_output><jupyter_text>3.2 Dictionaries So right now, we have a way of mapping a number (index) to a value (element) using lists. However, there are cases when we want to map pairs of values by something more complex than just the location in a list. We can do this through *dictionaries*, which map key-value pairs.For example, we might want to map students' names to their grades.<jupyter_code>grades = {}
grades["rosa"] = "30"
grades["pat"] = "92"
grades["andrew"] = "76"
grades<jupyter_output><empty_output><jupyter_text>Dictionaries function basically the same way that lists do, just with user-defined keys instead of indices as keys.Just like how lists can contain other lists, dictionaries can also contain other dictionaries! This leads to a tree-like dictionary structure, similar to how a real dictionary works – a chapter for 'A' words, and then within that chapter is each word starting with an 'A'.<jupyter_code>dictionary = {
    "a": {
        "aardvark": "looks like an anteater",
        "amazing": "me"
    },
    "b": {
        "baah": "sound sheep make",
        "bee": "stinging insects that are good for the environment"
    }
}<jupyter_output><empty_output><jupyter_text>Again, just like with lists, we can access the definition of `aardvark` by double-indexing:<jupyter_code>dictionary['a']['aardvark']<jupyter_output><empty_output><jupyter_text>3.3 For Loops We're going to take a step back and revisit the idea of control flow. Let's say we want to print each item in the list `x`. One way we can do that is using a counter variable and a while loop.**Practice**: Print out each element in the list `x` using a while loop.*Hint*: You might find the `len()` function helpful here  This works perfectly well, but we had to add extra lines of code and declare a variable that we won't be using outside of the loop. Another way we could implement this is by using another type of loop called a *for loop*. A for loop looks like this:    for {index variable} in {iterable}    For now you can think of an iterable as a collection of items that Python can look at individual elements of. Here's how we could achieve the same thing as above, but using a for loop.<jupyter_code>for el in x:
    print(el)<jupyter_output><empty_output><jupyter_text>Sometimes, you do want to iterate through the elements of a list by the index. We can still use an index variable, but instead of declaring one and incrementing it manually, we can use the power of for loops as well as a helpful function called `range()`. Range takes a number as input and basically outputs a list of that length with increasing integers starting from 0. (It actually outputs a `range` object, but in practice it is often used as a list in this way).    range(3) = [0, 1, 2]    In conjunction with `len()`, we can iterate through the indices of a list in order. **Practice**: Using `range()` and `len()`, print the elements of `x` in reverse order.<jupyter_code>## TODO<jupyter_output><empty_output><jupyter_text>You can also define a starting index for `range()`, which will give you an iterable starting from your starting index and up to (but not including) the second input.<jupyter_code>for i in range(2, 5):
    print(i)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08356953892668179, 'fraction_numerical': 0.009306500377928949, 'mean_word_length': 4.933015695067264, 'pattern_counts': {'":': 6, '<': 209, '<?xml version=': 0, '>': 207, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '50', 'dir': 'jupyter-structured-clean-dedup', 'id': '386022', 'license': 'MIT', 'n_tokens_mistral': 6227, 'n_tokens_neox': 5971, 'n_words': 2879, 'path': 'Lectures/Lecture2/intro/Intro to Python.ipynb', 'repo_name': 'alaymodi/Spring-2019-Career-Exploration-master', 'size': '21205'}