<jupyter_start><jupyter_text>Introduction to PythonMost aspects of Simons Observatory will require knowledge of Python. This notebook will give a "lightning" introduction to the Python programming language; we try to focus on the knowledge and skills that will give you the biggest bang-for-your-buck in your projects! That being said, we will cover just a fraction of what Python is capable of. If you want to learn more (or need to learn more) for your projects:* Don't hesitate to reach out to us the Simons-NSBP help_desk on slack with any questions! Just for reference, I (Zach) helped write this notebook -- I'm also in the Simons-NSBP slack workspace.* Don't hesitate to reach out to your project mentor(s). If they can't help with a specific question, they definitely can put you in touch with someone who can!* The Python documentation (https://docs.python.org/3/) is super user-friendly and has everything from tutorials to APIs for many modules (if you don't know what this means, hopefully you will by the end of this notebook)* Google is your friend. I google Python questions every 10 minutes when I'm writing a code. If your question is well-posed, 95% of the time someone has asked it already on the internet. Best websites: https://stackoverflow.com and https://stackexchange.com/ Just a couple more things before diving in:* We use Python 3 (the newest version of Python). If you are learning Python for the first time, stick to Python 3 (not 2). Specifically, Python 3.6, 3.7, or 3.8 will work well.* Python notebooks (what this is) are super helpful as teaching devices. They combine both text and code and allow any user of the notebook to change code and experiment on-the-fly. If you have to submit a report on something you code, consider building a notebook!* Most of the time, Python is *not* built in a notebook; rather, all of the code lives in a .py text file. For it to work on your computer (instead of Google's server), you'll need to install Python. You can install it directly (https://www.python.org/downloads/), but I prefer using a *distribution* (called "Anaconda", https://www.anaconda.com/products/individual), which packages Python together with various modules. As you'll see, "modules" are super useful, and there are hundreds. Anaconda is really good at managing/installing them, so you don't have to do that yourself. If you need help with that during your project, ask us or your mentor! Part 0 What is this?This is a Python notebook! It is an interactive, human-readable way of presenting Python code. A notebook is composed of "cells:" there are "Markdown" cells (like this one), which just contain helpful text. And there are "Code" cells, which actually contain the code, like this one:<jupyter_code>print('hello world')<jupyter_output>hello world<jupyter_text>To execute the code inside a code cell, make sure the cell is selected, then press shift + enter.Oh, and before we start, if you learn nothing else, remember this: the "print" statement is your friend in Python. If you want to debug any part of your code, or break it down to see how data evolves during the execution of your program, throw in print statements. In a code cell, if you type "print(something)", Python will include whatever is inside the parentheses in the output. That "something" could be text, or the value of some mathematical expression, or the name of a file, just to name a few examples! Part 1 The BasicsPython, like any programming language, has a set of rules for how to build the code. If you don't follow the rules, your computer won't know how to "read" your code; your computer will print an error and the code will stop running. One reason Python is so widely used is because its syntax -- its set of rules -- is very simple and flexible! Basic snippets of code even read like simple English:<jupyter_code># Everything after the hashtag is a comment - it just helps humans read and understand the code
# This code tests whether two variables are the same
# Try changing the value assigned to these variables to see what happens when you run this cell of code

my_number = 5
test_number = 7

if my_number is test_number:
    print('the two numbers are the same')

if my_number is not test_number:
    print('they are different')<jupyter_output>they are different<jupyter_text>Most of the magic happened in the line "if my_number is test_number". Hopefully you, as a human, are able to read this line and know what it's doing: testing whether two numbers are equivalent. Of course, not all code in Python is so directly "readable," but it's as user-friendly as a programming language can be.We won't be able to cover all the "rules" of Python, but the goal here is to learn the format of a simple code, and by example, how you could write something arbitrarily more complex. And if your code spits out an error (and this will happen, a lot, to the most experienced programmers), the great thing is it's OK: nothing actually broke -- you can go fix the mistake and try again.Let's explore what happened in the above code snippet. We'll be able to learn quite a lot, actually. 1.a Variables and TypesThe most fundamental aspect of any piece of code is assigning values to variables. Variables are just "names" that hold the value assigned to them. One rule of syntax is that the variable name must come first, before the value being assigned to it:<jupyter_code>my_number = 30<jupyter_output><empty_output><jupyter_text>If we switch the order, we get a "syntax error" -- we broke a rule:<jupyter_code>10 = my_number<jupyter_output><empty_output><jupyter_text>In the second case, because Python thinks the variable name must come first, Python tried to assign the value contained in "my_number" to 10. Of course, you can't change the value of 10! Keep in mind, Python is not angry with the fact the the value stored in "my_number" might not be 10. You will get the same error even if "my_number" actually already *is* 10. Rather, Python is angry with the code's *syntax*. The code is just not built according to "the rules," and the rule is the variable name comes first.A nice feature of Python is that variables can hold any *type* of value. There are many "primitive" types (meaning you get to use them when you install Python and only Python; other "modules," which as you will see are basically Python "expansion packs," can give you access to more data types), but we will focus on 4. You actually encountered 3 "primitive" types in the code snippet:* int (integer)* str (string)* bool (boolean)The first two might be more obvious:<jupyter_code>type(10)<jupyter_output><empty_output><jupyter_text>If we assign "5" to "my_number," asking Python for the type of the *variable* "my_number" will give the same result:<jupyter_code>my_number = 5
type(my_number)<jupyter_output><empty_output><jupyter_text>Great: your first Python type -- int -- lets Python know the value is an integer. Not too bad?Meanwhile, a string is basically just text. They are denoted by enclosing some text in quotes. So for instance:<jupyter_code>type('the two numbers are the same')<jupyter_output><empty_output><jupyter_text>There are many advanced operations you can do with strings (read the Python [documentation](https://docs.python.org/3/library/string.html) if you want to learn about them!), but the most basic is concatenation with a "+" sign:<jupyter_code>new_str = 'this has been a' + 'success?'
new_str2 = 'this has been a bigger ' + 'success!'

print(new_str)
print(new_str2)<jupyter_output>this has been asuccess?
this has been a bigger success!<jupyter_text>Watch out for spaces at the beginning and ends of strings!A boolean can only take on the values "True" or "False." Can you find where the boolean is hiding in our code snippet?<jupyter_code># Everything after the hashtag is a comment - it just helps humans read and understand the code
# This code tests whether two variables are the same
# Try changing the value assigned to these variables to see what happens when you run this cell of code

my_number = 5
test_number = 7

new_variable = my_number is test_number
print(new_variable)
type(new_variable)<jupyter_output>False<jupyter_text>Aha! It turns out that the piece of code "my_number is test_number" is itself a *boolean-valued expression*: it is either "True" or "False" (try changing the value of "my_number" to see what happens), and we can assign the *value* of this expression to a variable! So we could have written our code snippet like this:<jupyter_code># Everything after the hashtag is a comment - it just helps humans read and understand the code
# This code tests whether two variables are the same
# Try changing the value assigned to these variables to see what happens when you run this cell of code

my_number = 5
test_number = 7

# Assign the True/False value of the
# statement "my_number is test_number" to
# a variable "equality_test"
equality_test = my_number is test_number

if equality_test:
    print('they are the same')

if not equality_test:
    print('they are different')<jupyter_output>they are different<jupyter_text>Here's another boolean variable for you; hopefully this is crystal clear:<jupyter_code>my_variable = 5 < 3
print(my_variable)
type(my_variable)<jupyter_output>False<jupyter_text>The *boolean-valued expression* "5 < 3" evaluates to False. Python can calculate the expression's value (False) and store it in a variable, just like a number or a piece of text.One quick rule about writing boolean-valued expressions: if you're working with someone else's code, you will often see "==" in place of "is" and "!=" in place of "is not". If one of the values on either side of the comparison is not being presented to Python in the form of a variable, we should use "==" and "!=":<jupyter_code># Python is not happy if you do this
# because 7 is not a variable
#
# DISCLAIMER: this will only give you an error if you have Python 3.8 or newer!
# but, it's still "good practice" to use "==" when in doubt
my_number is 7
# Python is happier if you do this
my_number == 7<jupyter_output><empty_output><jupyter_text>And finally:<jupyter_code># Python is also perfectly happy if you do this
my_number == test_number<jupyter_output><empty_output><jupyter_text>Because "==" and "!=" work in more cases than "is" and "is not", let's start using them in our snippet:<jupyter_code>my_number = 5
test_number = 7

equality_test = my_number == test_number

if equality_test:
    print('hooray')
    
if equality_test != True:
    print('they are not the same')<jupyter_output>they are not the same<jupyter_text>There is one more primitive type of data: a "float", which is basically a number that isn't an integer:<jupyter_code>type(10.3)<jupyter_output><empty_output><jupyter_text>Floats are obviously extremely useful when using Python to perform some analysis of actual data, since data is rarely restricted to integer values. We will see why ints are useful in other ways in the next section.There are just a couple more quick "rules" you should know about these 4 types of data:<jupyter_code># Be careful when comparing floats and ints
this_works_as_expected = 10.3 > 5


# But watch out if we do this
my_number = 5
test_number = 5.0
this_doesnt_work_as_expected = my_number is test_number

print(this_works_as_expected)
print(this_doesnt_work_as_expected)<jupyter_output>True
False<jupyter_text>Clearly, Python has trouble when the value of a float and an int are the same. *You* know that 5.0 equals 5, but *Python* thinks otherwise. Why? Well, because 5.0 is a *float* and 5 is an *int* -- they are two different types, and different types cannot be the same* (\*another reason to use "==" instead of is: "==" will actually allow Python to correctly handle this special case, try it out!) Here's the real point: unless you *must* (for some reason) represent quantitative data as an int, it's probably best to just start it off as a float. Hmm, how would we change the type of some value, you might ask?Enter the second "quick rule," which is really more of a useful trick: you *can* change the type of some data. This is called "casting":<jupyter_code># Cast an int to a float
test_float = float(2)

print(test_float)
type(test_float)
# Cast a float to a str and concatenate it with another string
base_str = 'hello world, i am this many years old: '
str_to_add = str(99.99)

print(base_str + str_to_add)
type(base_str + str_to_add)
# Create a float by dividing two ints
new_float = 5 / 7   

print(new_float)
type(new_float)

# Or by dividing an int by a float
float_again = 2 / 1.
print(float_again)
type(float_again)<jupyter_output>2.0<jupyter_text>Congratulations! You are now a Python variable and primitive types professional!By the way, there is one rule that supercedes all others: if you are new to coding or Python, some of what we just did may still be confusing, and that is more than OK, that is basically *expected*. Anyone who seems like a programming whiz had similar moments of confusion when starting out, and everyone continues to learn only through practice. Definitely *take the time for yourself* to learn by changing up the above code cells and just seeing what happens. And if you still have questions, absolutely don't hesitate to reach out to me (Zach) or anyone at the help_desk. 1.b OperatorsMost of your time in Python will be spent manipulating quantitative data. I would be remiss if I did not show you the rules for basic mathematical operations!Hopefully these need no explanation, so let's just run through them. The key takeaway is the *specific character(s)* you need to type to get the behavior you want:<jupyter_code># Addition (+)
2 + 2
# Subtraction (-)
5 - 79
# Multiplication (*)
10*30
# Division (/)
7/2
# "Integer" division (//)
7//2<jupyter_output><empty_output><jupyter_text>Integer division is basically like rounding the result down the result to the nearest integer. Anywho, onwards:<jupyter_code># "Modular" division (%)
7%2<jupyter_output><empty_output><jupyter_text>Modular division gives you the *remainder*. And finally:<jupyter_code># Exponentiation
3**2<jupyter_output><empty_output><jupyter_text>1.c Lists and ObjectsEarlier I said that variables can hold any type of value, and we just examined 4 "primitive" types that come along with Python. But variables can also hold *objects*.In programming, an object is basically an organized set of values. So, you might have a data table, with many rows and columns, which you want to store in your program and refer to with a single variable name. Or you might have a list of people to invite to your birthday party, but rather than making a new variable for each person, you want to refer to the entire list with a single variable name. The possibilities for objects are endless (the best part: you can [create your own](https://docs.python.org/3/tutorial/classes.html)! But that's an advanced topic you could explore on your own time...)We'll focus on a key object type that comes standard with any installation of Python: a *list*. Again, modules (those "Python expansion packs" we haven't looked at yet) may give you the option of using a *huge* number of other object types. Understanding how lists work, though, will (a) be widely useful on its own in your projects this summer, and (b) provide a basis for you to understand any fancier objects later on.Just as with the "4" primitive value types, we assign *lists* to a variable name by writing the variable name first:<jupyter_code>my_first_list = ['one', 'two', 3, 4.0, my_number, True]

print(my_first_list)
type(my_first_list)<jupyter_output>['one', 'two', 3, 4.0, 5, True]<jupyter_text>That was...exciting? Let's deconstruct the code.As before, "my_first_list" is just a variable name, available for use later on in your code. We told Python to construct the list by typing the pair of brackets, "[ ]". We just happened to fill the list with a few values and variables, each separated by a comma (and each of an arbitrary data type). And, well, that's all there is to it: a list is an ordered set of values/variables, of any type, enclosed in brackets and separated by commas.Try doing the next code cell by yourself!<jupyter_code># Make a list of 3 friends you will invite to your birthday
names = 

# Make a list of their ages
ages =<jupyter_output><empty_output><jupyter_text>Though simple, lists can be immensely useful. For one thing, not only can they hold any value type, they can even hold objects, including other lists:<jupyter_code># Let's put the names and ages of our friends together
friend_info = [names, ages]

print(friend_info)<jupyter_output>[['a', 'b', 'c'], [1, 2, 3]]<jupyter_text>This could be extremely useful if, say, you have some quantitative data that lives in two or more dimensions, like a scatter plot. Lists are also "ordered," which means you can access each of the values/variables/objects stored inside in a universal way -- you don't have to know the variable name of anything you put in the list! The process of accessing or manipulating entries in a list is called "indexing:" the "index" of an item in a list basically means its location. Let's look at an example:<jupyter_code># Retrieve the first name on your list of friend names
first_name = names[0]

print(first_name)<jupyter_output>a<jupyter_text>Remember, "names" is a list containing whatever you put into it earlier. We access items in the list by placing another pair of brackets immediately after this list: names[0]. Don't be alarmed! This pair of brackets is *not* a list -- rather, it's telling Python that we want to access index "0" of the object ("names") *immediately* preceding it. And yes, as you can infer, Python indeces start with "0" and count up. So if we wanted the third age in your list of friends' ages:<jupyter_code># Retrieve the third age on your list of friend ages
third_age = ages[2]

print(third_age)<jupyter_output>3<jupyter_text>Trust me: the more time you spend in Python, the more you will think the "right" thing to do is to *always* start counting up from 0...So here's the rule with lists: to make them, place any value, or variable (whether or not that variable holds a value, or an object, which might itself be a list!) inside brackets, and separate them by commas. To *access* an item in a list, type a pair of brackets with the *index* of the item you want to retrieve inside, *immediately* following the list itself.Hopefully this last example makes things clear. It will look a little tricky at first, and if it confuses you, play around by adding some intermediate steps, or reach out for help!<jupyter_code># Using only the combined list "friend_info", change the name
# of your second friend to "Kasey". Do it in only one line of code!

friend_info[0][1] = 'Kasey'

print(friend_info)<jupyter_output>[['a', 'Kasey', 'c'], [1, 2, 3]]<jupyter_text>Here's the trick: Python reads from left to right, and will try to interpret the code as it goes. So the first (non-trivial) expression Python finds is "friend_info[0]". Based on our definition of the list "friend_info," the first item inside is another list: "names". Try it out for yourself:<jupyter_code># Change Kasey's age to 30.65, using only the combined list
# "friend_info" and one line of code

# your code goes here!

print(friend_info)<jupyter_output>[['a', 'Kasey', 'c'], [1, 2, 3]]<jupyter_text>There are three more super useful tricks you should learn about lists:<jupyter_code># We can get the length of a list with the "len" function
# Place your list in the parantheses (just like a mathematical function!)
num_names = len(names)
print(num_names)<jupyter_output>3<jupyter_text>and, we can quickly build lists that contain an ordered set of numbers:<jupyter_code># Get all the numbers from 0, up to but not including 50, in a list

zero_to_49 = list(range(50))
print(zero_to_49)<jupyter_output>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]<jupyter_text>Two things happened here: we used the "range" function -- which generates integers from 0 up to *but not including* the integer in the parentheses (see the Python [docs](https://docs.python.org/3/library/functions.htmlfunc-range) for more)! But "range" does not place the numbers in the format of a list -- to put the result into a list, we *cast* the result to a list *type*, just like we did with the primitive types in 1.a.Finally:<jupyter_code># We can merge two lists together
# This is called concatenation, like with strings!
one_list = ['a', 'b', 'c']
another_list = [1, 2, 3]

final_list = one_list + another_list

print(final_list)<jupyter_output>['a', 'b', 'c', 1, 2, 3]<jupyter_text>To be clear: both objects being concatenated must be lists -- even if you wanted to add a single number to the end of a list:<jupyter_code>numbers_list = another_list + [4]
print(numbers_list)<jupyter_output>[1, 2, 3, 4]<jupyter_text>1.d.1 Flow Control: if/else statements As you build more and more complicated programs, you will often find you want your program to do something different: (a) depending on some condition that (b) you want your code to figure out for itself.In fact, we already saw this, in our first code snippet! Perhaps it's best to learn the relevant rules by example. Let's bring back our code snippet from part 1.a:<jupyter_code>my_number = 5
test_number = 7

equality_test = my_number == test_number

if equality_test:
    print('hooray')
    
if equality_test != True:
    print('they are not the same')<jupyter_output>they are not the same<jupyter_text>You may have already discovered that the output of this code is controlled by whether my_number and test_number are the same, but if not, try changing the value of my_number so that it is the same as test_number and run the cell. Next, change the values of my_number and test_number so that they are different, and run the cell. Perhaps it's a bit simple, but the code does something *different* depending on whether or not the boolean variable "equality_test" is true. This is basic flow control!What are the rules to doing this correctly? Evidently, the control statement starts with an "if". Next comes a boolean expression (stored in the variable "equality_test"). The line ends with a colon, ":". But there could be hundreds of lines of code left in our program -- how do we tell which lines Python will run *only* if the condition is true?This is where Python *formatting* comes in. We tell Python which code to execute *if equality_test is True* by *indenting it immediately after the colon*. *All* the indented code will *only* be executed if the *boolean expression* in the "if statement" is true:<jupyter_code>if 2 < 3:
    print('the code')
    print('has gone')
    print('down this')
    print('path')

print('the code always does this')

if 2 > 3:
    print('the code')
    print('will not')
    print('do this')
    
print('the code always does this, too')<jupyter_output>the code
has gone
down this
path
the code always does this
the code always does this, too<jupyter_text>If you don't indent immediately following the if-statement, Python will say you broke another rule:<jupyter_code>if True:
print('i forgot to indent')<jupyter_output><empty_output><jupyter_text>You should learn two "tricks" that can help construct intricate code flow with if statements. The first is the "else" statement. The "else" statement is built very similar to an if-statement, but without the boolean expression. Instead, each else statement is "married" to the immediately preceding if-statement at the *same level of indentation*. The else-statement is synonymous with "Python, do this instead when the if-statement was False":<jupyter_code>a = 4
b = 3

if a == b:
    print('they are the same')
    
else:
    print(a, b)
    print(a == b)
    print('they are different')<jupyter_output>4 3
False
they are different<jupyter_text>The second trick is that if-else statements can be *nested*. Keep track of your indentation!<jupyter_code>c = 10
d = 10

e = 11
f = 12

if c == d:
    if e == f:
        print("c is d, and e is f")
    else:
        print("c is d, but e isn't f")
else:
    if e == f:
        print("c isn't d, but e is f")
    else:
        print("c isn't d, and e isn't f")<jupyter_output>c is d, but e isn't f<jupyter_text>Take some time to follow where the code went in this snippet, and why (and reach out for any help if you need it!). 1.d.2 Flow Control: for loopsA second, critically important type of flow control is a "for" loop. For loops are used to iterate through items in a sequence and execute *the same code* with each new item. This sounds a little abstract, so let's look at an example:<jupyter_code># Tell each one of our friends they are invited to our birthday party
for n in names:
    print(n + ", you are invited to my birthday party")<jupyter_output>a, you are invited to my birthday party
Kasey, you are invited to my birthday party
c, you are invited to my birthday party<jupyter_text>The code structure is *similar* to if/else statements. We start with a "for", then we introduce a *dummy variable*; in this case, "n". Next we add "in", and finally, we must add the name of an *iterable object*. An iterable object is any object that supports iteration over its elements -- for the purposes of this notebook, we can just use a list: our old list from part 1.c, "names".As before, a colon (":") signifies the end of the for-statement, and then we have an indented block of code. A for-loop will execute this, and only this, *indented* block of code over and over; each time the *dummy variable* defined in the for-statement will take on the *next value in the list*. Here's another example, using another old list, "zero_to_49":<jupyter_code># print the multiples of 8 from 0 and up to, but not including, 50
for i in zero_to_49:
    if i % 8 == 0:
        print(str(i) + ' is a multiple of 8')<jupyter_output>0 is a multiple of 8
8 is a multiple of 8
16 is a multiple of 8
24 is a multiple of 8
32 is a multiple of 8
40 is a multiple of 8
48 is a multiple of 8<jupyter_text>Once we understand this piece of code, we are well on our way to some reasonably complicated programming: it combines everything we've talked about so far! Let's go piece by piece; be sure to digest each idea before moving on...Python will execute the code inside the for-loop -- which is itself an if-statement! -- *for each value of i*. From the definition of "zero_to_49", we know those values of "i" will be (in order) "0, 1, 2 ... 49". The if-statement says "if the *remainder* of i divided by 8 is 0", while the print statement (a) first casts an *int* ("i", whatever it may be) to a *str*, and then concatenates that string with a message. That's a lot of information in such a small piece of code -- hopefully you can start to feel just how powerful Python could be for data analysis!Are you ready for another coding exercise?<jupyter_code># Use a for loop to add up the total age of your friends
# Hint: a = a + b will overwrite the value of "a" with
# the previous value of "a" plus the value of b

# your code goes here<jupyter_output><empty_output><jupyter_text>Remember, if you need any help, or just want another hint, reach out to Martina, Joe, or Zach! Just because it's a super common practice, we should look at a specific for-loop setup: looping over list indeces. Oftentimes it is the case that you will need to access each item in a list by its *index*, perhaps to modify it in place. We can do this easily with a for loop. Here's an example: imagine you realized you entered the ages of your friends wrong, and you want to add a year to the first person, two years to the second, three to the third...I am going to throw in three tricks here which we'll deconstruct after, but see if you can't figure out how they work by either examining the output or playing with the code:<jupyter_code># Add "n" years to the age of your "nth" friend

# Loop over all age *indeces*
for n in range(len(ages)):
    years_to_add = n + 1
    ages[n] += years_to_add
    
# Show the new ages of your friends
print(ages)<jupyter_output>[8, 16, 24]<jupyter_text>First, we encountered the "range" function from part 1.c, only (unlike the variable "zero_to_49") this time we did *not* cast its output into a list first. You *will* see this done in other code, and other people reading your code *will* expect it to look like this. What's going on here? Well, the result of "range" is an *iterable object* but *not* a list -- all lists are iterable objects, but not all iterable objects are lists! For the purposes of learning the Python ropes, using "range" in your code *without* casting the result into a list will only work in this specific case: inside the for-statement.Second, we didn't have to explicitly enter the argument of the range function like we did when building the "zero_to_49" list. Instead, we let Python figure it out by using the "len" function, which we also met at the end of part 1.c. So putting it together, inside the for-loop, the *dummy variable* "n" will start at 0 the first time through the code block, then it will be 1 the second time through...and the last time through, it will be the length of the list "ages" minus 1. *Here's the key:* remember that Python starts counting indeces *from zero*, so "n" will iterate through *all the indeces* in the list "ages"! The last trick is the operator "+=", which is just a shorthand way of writing the "hint" from the for-loop coding exercise. It says "take the variable on the left-hand-side ("ages[n]") and *add to it* whatever is on the right-hand-side! You can also use "-=", "*=", "/="...Let's pause. Great job! With these pieces -- variable types, mathematical operations, lists, if-else statements, and for-loops -- you are well-equipped to tackle a *huge* number of possible programming tasks!Perhaps it would be useful to organize the topics, rules, and tricks we covered so far. This way, you can remind yourself of what you feel more comfortable with, and what you might want to go back and review: Part 0 What is this?* Highlight a code cell and press shift + enter to execute its code* print(some expression) will display the value of whatever is in the parentheses  Part 1.a Variables and Types* Assign values to variables by writing "variable = value," in that order* We covered 4 *types* of primitive values: int, str, bool, and float* In writing *boolean-valued* expressions, write "==" to mean "is the same as" and "!=" to mean "is not the same as"* You can change the type of some value/variable by "casting" it to another type. This can happen explicitly -- int(10.3) is an int -- or implicitly -- 7 / 2 is a float Part 1.b Operators* We covered +, -, \*, /, //, and \*** We talked about the +=, -=, \*=, and /= operators as well Part 1.c Lists* We can build lists by enclosing variables in brackets, separated by commas* Lists can be "multi layered:" you can have a list of lists!* We can access the items in a list by indexing: place the index number in brackets immediately after the list. Remember, the first index is "0"!* If you have a list of list, you can compactly "stack" your indexing, e.g. list[0][4] returns the 5th item in the first list* We gave an overview of the "len( )" and "range( )" functions* Concatenation of two lists is achieved with the "+" symbol Part 1.d.1 Flow control: if/else statements* We write if-statements like this: "if *condition*:" followed by an indented block of code. *Condition* must be boolean-valued* An else-statement can follow an if-statement *on the same level of indentation*. Code following an else-statement will execute when *condition* is false* if/else-statements can be nested Part 1.d.2 Flow control: for loops* We write for-loop-statements like this: "for *dummy variable* in *iterable object*:" followed by an indented block of code* The for-loop repeats the indented block of code, updating *dummy variable* each time for each item in the *iterable object** Lists are a widely-useful example of an *iterable object** The "range( )" function can also be used directly as an *iterable object** It is common to see for-loop statements written like: "for *dummy variable* in range(len(*list*)):"The next two sections are not nearly as long as "The Basics," but they'll each introduce absolutely *critical* tools for your programming toolbelt, both this summer and beyond. Part 2: Defining FunctionsWhen using Python for data analysis, we will often want to perform the same manipulation on our data, over and over again. Rather than retyping the same code every time, we can package pieces of code -- each performing a specific task -- into a *function*.In some sense, this is just like the functions you are used to seeing in math and physics. For example, consider a particle falling from a initial height $h_0$, with initial velocity $v_0$, and we want to know its height at time t. We know the answer is: $h(t) = -0.5gt^2 + v_0t + h_0$. But rather than write the complicated expression on the right-hand-side every time we want to refer to the height of the particle, we usually just write $h(t)$. A *function* is a shorthand that goes from input(s), like $t$ (and also $v_0$ and $h_0$), to output(s), $h(t)$.We can encode the exact same shorthand in Python:<jupyter_code># Write a function that gives the particle height
# at time t, for a given v_0 and h_0

def h(t, v_0, h_0):
    g = 9.81 
    return -0.5 * g * t**2 + v_0 * t + h_0    

# Use this function to model an object falling from 100 meters at rest
t_final = 3
h_final = h(t_final, 0, 100)
print(h_final)<jupyter_output>55.855<jupyter_text>Let's deconstruct the code!All functions begin with *definition* statements, which tell Python "pay attention, we are about to define a function!" The Python rule is: we do this by proclaiming "def", followed by a space. *Whatever* text immediately comes after the space will be the *name* of the function. If you want to *use* this function later in your code, you will type this name at that time.We enclose the *arguments* of the function in parentheses, separated by commas. The parentheses *immediately* follow the function name. Arguments are variables that the code in the function requires in order to run. Perhaps this is the most subtle point in defining a function: whereas in physics you may be happy to write $h(t)$, in Python, you must explicitly write all of the arguments that will determine the value of the function, i.e. $h(t, v_0, h_0)$. When defining functions in your own code this summer, you may find yourself spending more time than you would think determining exactly which information is actually needed as an input!As with flow control, the def-statement concludes with a colon, ":", and the code that will be executed *whenever* the function is called later on *immediately* follows the colon in an indented block. At this point, you can implement any code you like! The key is that this indented code will have access to the variables in the arguments *at the time the function is called*. In our example, when we called our function to model the falling particle, we passed the values in "t_final", 0, and 100 to our function. Because of the *order* of the arguments in the def-statement, when we *called* the function Python assigned the value in "t_final" to t, "0" to v_0, and "100" to h. To end the function, in particular if you would like the function to have some *output*, you use a "return" statement. Any Python expression -- it could be a value, variable, object, or a mathematical expression like in the example -- can go in the return statement, it just has to follow the word "return" with a single space in between.Anyway, those are the boring Python rules. Since you are well-acquainted already with functions in the abstract, let's try an exercise! Absolutely don't hesitate to reach out with any questions.<jupyter_code># First, let's practice using functions in the context of Python code
# Give a particle an initial height and velocity by assigning those values to two variables

# your code goes here

# We want to find the height of the object at several times
# Make a list of times
# Remember, there are many ways you could build your list!

# your code goes here

# We want to place the output heights in another list
# Either build a list of the same length as your list of times
# Or initialize an empty list

# your code goes here

# Use a for-loop and our function "h" to fill the output list
# with the heights of the particle at each time in your list of times
# Hint: if your output list is an empty list, you could concatenate
# it with the output of each function call!

# your code goes here

# print the list of times and the list of heights

# your code goes here<jupyter_output><empty_output><jupyter_text>Unlike in math and phyics, functions in Python are not restricted to quantitative data. Let's write a function which will output the age *and* name of your oldest friend:<jupyter_code>def age_name_oldest_friend(list_of_ages, list_of_names):
    
    # These variables will be updated
    # It's important that oldest_age_so_far starts as something
    # less than the youngest possible age!
    oldest_age_so_far = -1
    oldest_index = -1
    
    # Loop through your list of ages
    # The goal will be to find the index corresponding to your oldest friend
    for i in range(len(list_of_ages)):
        
        # If the age of your friend at this index is
        # older than the oldest_age_so far, do something
        if list_of_ages[i] > oldest_age_so_far:
            
            # Save this index as the new *potential* index of your oldest friend
            oldest_index = i
            
            # Update the oldest_age_so_far to be the age of your friend at this index
            oldest_age_so_far = list_of_ages[i]
    
    # We are back to the same indentation level as the for-statement, so
    # this code is no longer part of the for-loop
    # Return the age and name of your oldest friend
    return list_of_ages[oldest_index], list_of_names[oldest_index]

# Try it out! Lets start with a new list of friends and their ages
ages = [20, 23, 25, 22]
names = ['w', 'x', 'y', 'z']

# Call the function with the arguments in the right order
# Assign the outputs to two variables, also in the right order
oldest_age, oldest_name = age_name_oldest_friend(ages, names)

print("oldest age is " + str(oldest_age))
print("oldest name is " + oldest_name)<jupyter_output>oldest age is 25
oldest name is y<jupyter_text>Understanding the *logic* of how this function works is *not* easy. If you find it confusing, try "following the intructions" of the code by hand for a small test case, or add some print statements in function code to trace what happens as the code runs. But the *key* point I want you to take away is that most of the inputs and outputs were *not* quantitative. Both of the arguments are *lists*. One of the outputs is a *string*. Functions in Python are useful for manipulating *all types* of your data, including data that lives in an object, like a list.By the way, we just learned a new trick: if you want a function to have *multiple* outputs, separate them with commas in the return statement. Just make sure when you actually *call* the function later on in your code, you have the correct *number* of variables (separated by commas) on the left-hand-side of the variable assignment equation, in the *correct* order!It's time for another coding challenge! This time, you will write your own functions. Try to make as much headway as you can by yourself, but please reach out for *any* and *all* help:<jupyter_code># Write a function that calculates weighted-averages
# Its arguments should be a list of numbers to average, and
# a list of weights to apply to those numbers

# your code goes here

# Write a function that calculates the "center of mass"
# of a physical object
# Treat the "index" of a list as the "position", and the value at that
# position as the mass at that position
# We give you the mass as a function of position in the following list
# Hint: in this new function, use the function you just made!

mass_by_position = [0.2, 1, 1.5, 1.5, 2, 5, 6.2, 6., 4, 2]

# your code goes here

# Print the result. Does it make sense?<jupyter_output><empty_output><jupyter_text>One last note about functions: sometimes, you will want to give your arguments a default value. That way, you won't have to explicitly specify what value an argument should be every time you use your function. We do this by writing "argument = *default*" in the def-statement, as opposed to just "argument" between the parentheses:<jupyter_code>def default_example(arg1 = 10):
    print(arg1)
    return

# Try it
default_example()

# Try it again
default_example(arg1 = 40)<jupyter_output>10
40<jupyter_text>The first time we use the function, we passed no arguments. Instead, Python knew to use the default value of 10. The second time, we pass a value of 40, but we need to respect the function definition: if the def-statement uses this feature of Python, you'll need to explicitly follow the same format when you *use* the function.  Part 3: ModulesWe've mentioned "modules" a few times so far, referring to them as "Python expansion packs." Now it's time we learn what they *really* are. Since you have gotten a taste of how useful functions can be in Python, you'll be excited to learn that modules are "libraries" of custom functions that other people have already written, and you get to use them in your code (see the Python [docs](https://docs.python.org/3/tutorial/modules.html) for more).This is great, because it allows us to write compact, modular code. We don't ever need to write functions that calculate weighted averages by hand, because someone has already done it for us. All we need to do is *load* their *module*.We will explore what this looks like in the context of [NumPy](https://numpy.org/doc/stable/reference/index.html) (a.k.a. "numerical Python"), which you will probably want to use in 99.9% of all Python code you ever write. Basically, numpy contains thousands of functions built around a new *type* of object, called a "numpy array." Numpy arrays are glorified lists, but they are so much more powerful, as we'll see.We also mentioned "APIs" in the context of modules. As promised, we can define an API now. An "application programming interface" is just the set of verbal instructions so that you know how to use functions in a module. For example, if we want to use numpy's "sort" function, we can just look at the [online API](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.sort.htmlnumpy.ndarray.sort). The webpage displays the order of the arguments in the function call, what each argument means, what is returned by the function, and some examples on how to use it. Usually, the fastest way to find the API is to Google it: e.g., google "numpy sort" and you should be able to find the webpage I linked. 3.1 Importing modulesIn order to use the numpy module, we need to "bring it in" to our code. This means two things: (a) we need to install numpy, and (b) we need to know the Python "rules" for actually importing the numpy functions into our code. The google colab environment already has numpy installed, but if you plan on using Python locally, you'll need to install it on your computer. We won't do that here, but please follow up for help if you'd like to do that. How do we import numpy into our code? It's very exciting (that's sarcasm):<jupyter_code>import numpy<jupyter_output><empty_output><jupyter_text>Python is clever, though, and programmers often write the following instead:<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>This is Python's way of "storing" the module name in something that's more convenient to type (kind of like a variable). Now when we type "np", Python will interpret that as "numpy".Let's start using one of numpy's functions, for example, the "sort" function. We'll sort a list of arbitrary numbers:<jupyter_code># Some random list
test_list = [2, -100, 3, 4.3, 3.14, 9]

result = np.sort(test_list)

print(test_list)<jupyter_output>[2, -100, 3, 4.3, 3.14, 9]<jupyter_text>Let's break it down! Evidently, the "sort" function is accessed by adding "." and the function name to the module, "np". Of course, this generalizes to a rule: *module.function* is how we access *any* function in *any* module that we have imported into our code.Modules, including numpy, often have "subtructure," by which I mean they will often have layers of modules before you get to the actual functions. For instance, if you wanted to generate 10 random samples from the mean-0, unit-variance normal distribution:<jupyter_code>gaussians = np.random.randn(10)

print(gaussians)<jupyter_output>[-0.82031885 -0.5785891   0.11910147 -0.43089493 -0.79887988 -0.86408351
  1.41837786  1.00199106  0.86114468 -0.47645633]<jupyter_text>So here, "random" is a submodule that lives in numpy. And inside of random, we find the "randn" function (see [docs](https://numpy.org/doc/stable/reference/random/generated/numpy.random.randn.html) for the API), available for our use. In other words, the rule for accessing functions in modules can also look like: *module.module.function*, and so on, depending on how the developers of numpy organized their code.Sometimes you will only want to import specific submodules or functions, rather than the entire available library. We use a slightly modified import statement for that:<jupyter_code>from numpy import sort as s
from numpy import random as r

print(s(test_list))
print(r.randn(10))<jupyter_output>[-100.      2.      3.      3.14    4.3     9.  ]
[-0.93697847 -0.61567598 -0.09763264 -1.10419179 -1.16017902  0.72097572
 -0.47821279 -0.38336773  0.71514722  0.67816836]<jupyter_text>To specify a specific function/submodule, the rule is: "from *parent* import *the thing you want*"; our trick of renaming modules still applies, and even to functions (we changed the name of numpy.sort to "s")! 3.2.a Numpy arrays: constructionNumpy's power comes not only from the massive number of useful functions it makes available, but from a new *object type* that it defines: the numpy "ndarray", or array for the purposes of explanation. The result of a numpy function is very often such an array:<jupyter_code>print(type(result))
print(type(gaussians))<jupyter_output><class 'numpy.ndarray'>
<class 'numpy.ndarray'><jupyter_text>Arrays are the workhorses of manipulating quantitative data. You'll gain a lot of experience working with them in the next notebooks in this course, so we won't spend too much time here, but it's useful to take a look at the basics.We learned one way of making an array: they are almost always the output of a numpy function! But we can build them from scratch, too (here's a [full list](https://numpy.org/doc/stable/reference/routines.array-creation.html) you can explore on your own time if you want, but of course, no need to do so). Let's look at a few very common functions that will do this for us:<jupyter_code># Cast a list to an array
test_list_arr = np.array(test_list)
print(test_list_arr)

# Create an array of all zeros, with size 5
zeros_arr = np.zeros(5)
print(zeros_arr)

# Create an array of all ones, with size 5
ones_arr = np.ones(5)
print(ones_arr)

# Create an array of all "your_number", with size 5
your_number = 3.1415
your_arr = np.full(5, your_number)
print(your_arr)

# Create an array of all the integers from 0 up to, but not including, 50
zero_to_49_arr = np.arange(50)
print(zero_to_49_arr)

# Create an array of all the numbers betwen 1.5 and 5.5 inclusive, of size 5, equally spaced
last_arr = np.linspace(1.5, 5.5, 5)
print(last_arr)<jupyter_output>[   2.   -100.      3.      4.3     3.14    9.  ]
[0. 0. 0. 0. 0.]
[1. 1. 1. 1. 1.]
[3.1415 3.1415 3.1415 3.1415 3.1415]
[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
 48 49]
[1.5 2.5 3.5 4.5 5.5]<jupyter_text>Hopefully, with your Python experience so far under your belt, the code (and the results) are fairly intuitive in terms of what's going on, but if not, please, reach out for questions! If you want more info, say on the "arange" function specifically, try also Googling "numpy arange" -- in addition to the API, you'll find other explanatory articles with examples, etc. I do this all the time while coding for help! 3.2.b Numpy arrays: axesLet's discuss one last aspect of numpy arrays before concluding: axes. Remember how I said numpy arrays are like "glorified lists?" Well let's see if this doesn't impress you!Let's say we wanted to represent some data in two dimensions, for example a digital image. This image is simple: the value at every pixel is 30. The number of pixels in one direction is 4, and in the other direction is 8. Watch this:<jupyter_code># Create a 4x8 image where every pixel is 30
shape = [4, 8]
image = np.full(shape, 30)
print(image)<jupyter_output>[[30 30 30 30 30 30 30 30]
 [30 30 30 30 30 30 30 30]
 [30 30 30 30 30 30 30 30]
 [30 30 30 30 30 30 30 30]]<jupyter_text>The API of the [numpy full function](https://numpy.org/doc/1.18/reference/generated/numpy.full.html) tells us that the first argument of "full" is the *shape*, and the second argument is the *fill_value*. We set the shape to be 4 *along the first axis of "image"* and *8 along the second axis of "image"* directly in the function call. What does this all mean? I think we'll learn best by looking at examples, rather than having me "tell" you.Our array "image" looks kind of like a list of lists, but I want you to start thinking more in terms of arrays and less in terms of lists -- this is more like an *array of arrays*: in some sense, there are *4 arrays*, each of length 8, living together in *image*. Just like lists, we can access them individually and modify them:<jupyter_code># Add 5 to the first row in image
image[0] += 5

print(image)<jupyter_output>[[35 35 35 35 35 35 35 35]
 [30 30 30 30 30 30 30 30]
 [30 30 30 30 30 30 30 30]
 [30 30 30 30 30 30 30 30]]<jupyter_text>Unlike lists, we can access particular elements across multiple "dimensions" *in one indexing* operation:<jupyter_code># Subtract 2 from the pixel in the fourth column of the second row
image[1, 3] -= 2

print(image)<jupyter_output>[[35 35 35 35 35 35 35 35]
 [30 30 30 28 30 30 30 30]
 [30 30 30 30 30 30 30 30]
 [30 30 30 30 30 30 30 30]]<jupyter_text>Whoa, fancy! Let's make sure we breakdown what happened here.Just like with lists, we access data in the array by typing the *index* of the data in brackets, immediately following the variable name. But unlike lists, a numpy array is smart enough to be "aware" of the fact that it is multi-dimensional, so you can specify both indeces in one pair of brackets, separated by a comma. This is exactly analogous to specifying a point on the plane with reference to both $x$ and $y$.Numpy refers to the possible-multiple dimensions of an array as "axes". The first axis is the "outermost array," or the "array of arrays". In this case, the second and last "axis" refers to each subarray, or "row", or the "innermost" arrays. When indexing the data, it respects this ordering of its axes; likewise, this is the order we specified when we passed \[4, 8] to the shape of our array.A trick in Python indexing is you can use the colon, ":", as *an index* -- this means "across all elements" in this axis. So if we wanted to change a full *column* of our array:<jupyter_code># Subtract 20 from the 7th column of image
image[:, 6] -= 20

print(image)<jupyter_output>[[35 35 35 35 35 35 15 35]
 [30 30 30 28 30 30 10 30]
 [30 30 30 30 30 30 10 30]
 [30 30 30 30 30 30 10 30]]<jupyter_text>By placing ":" in the first axis position, we told Python to return the "slice" of the data across all elements in the first axis. In particular, *we told it to return the item in the 7th position along the second axis, across all positions in the first axis.* Please let that statement digest for a few minutes if it isn't immediately clear!Thinking about axes is incredibly useful, because many numpy functions will take advantage of them. Let's first refresh our image:<jupyter_code># Create a 4x8 image where every pixel is 1
shape = [4, 8]
image2 = np.ones(shape)
print(image)<jupyter_output>[[1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1.]]<jupyter_text>We'll use the numpy "sum" function (API [here](https://numpy.org/doc/1.18/reference/generated/numpy.sum.html)) to give an example:<jupyter_code>sum_over_first_axis = np.sum(image2, axis = 0)
print(sum_over_first_axis)<jupyter_output>[4. 4. 4. 4. 4. 4. 4. 4.]<jupyter_text>Remember, our first axis is the "outermost" layer, the rows. By summing across *only* the first axis, we sum "across the rows". Likewise, we can sum across the columns:<jupyter_code>sum_over_second_axis = np.sum(image2, axis = 1)
print(sum_over_second_axis)<jupyter_output>[8. 8. 8. 8.]
{'fraction_non_alphanumeric': 0.07228870324377211, 'fraction_numerical': 0.022333860164070143, 'mean_word_length': 4.397481466436478, 'pattern_counts': {'":': 9, '<': 201, '<?xml version=': 0, '>': 201, 'https://': 15, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '59', 'dir': 'jupyter-structured-clean-dedup', 'id': '164351', 'license': 'MIT', 'n_tokens_mistral': 16030, 'n_tokens_neox': 14749, 'n_words': 8271, 'path': '1_Python_Intro.ipynb', 'repo_name': 'simonsobs/s-nsbp', 'size': '53148'}