<jupyter_start><jupyter_text>Day 2  Topics1. Bit of a review1. For-loops1. If-then statements1. Examples using `matplotlib`  Bit of a reviewThis is markdown!  There are cool things we can do here!* here I am making a list* **in bold!** We also talked a bit about variables as well and their types:<jupyter_code>variable1 = 1
variable2 = 3

variable_sum = variable1+variable2
print(variable_sum)
print(type(variable_sum))<jupyter_output><class 'int'><jupyter_text>We also played around with strings:<jupyter_code>myString = "hey I'm a string!"<jupyter_output><empty_output><jupyter_text>We also talked a bit about how to "slice them" by indexing and that indexing starts at 0:<jupyter_code>print(myString[0:5])<jupyter_output>hey I<jupyter_text>We also talked a bit about two ways to store lists of things, the first being lists which are native to Python:<jupyter_code>myList = [5, 6, 'boop!']
print(myList)<jupyter_output>[5, 6, 'boop!']<jupyter_text>And also NumPy arrays, which we can use if we import a separate library:<jupyter_code>import numpy as np
myArray = np.array([5,6.7, 8])
print(myArray)<jupyter_output>[5.  6.7 8. ]<jupyter_text>Which allows us to do mathy-type things like add 5 to all elements:<jupyter_code>print(myArray+5)<jupyter_output>[10.  11.7 13. ]<jupyter_text>BACK TO SLIDES FOR SOME INFO ABOUT for-loops & if-then statements! For-loopsFor reference: [Loop lecture, IS452](https://github.com/elliewix/IS-452-Spring2020/blob/master/Lectures/Week-02-ExpressionsAndLoops.ipynb)We are about to repeat stuff!  Programming making things more efficient!  Woo! While there is a single syntax rule for for loop, there are many types. We're going to start off with the three core patterns. We'll learn more patterns as we move forward, but all of those are tweaks or expansions off of one of these three core patterns.There are three kinds of loops:1. unpack some sequence of content and loop over those elements one at a time   * your input is the sequence, python will unpack it for you1. repeat a task some arbitrary number of times   * your input is the number of times1. loop over something for the purposes of looking up another thing   * this one is strange, where you provide the sequence of items that represent the lookup key for another item. These keys may be just numbers counting up, or they may be other kinds of content in a sequence (such as letters or words).   There three uses will have a very similar syntax and can be difficult to tell the difference between. For the sake of names, let's call them:1. an unpacking loop1. a range loop1. a reference loopWe're going to explore and reinforce our understanding of data types and other stuff as we play with loops.Sometimes you'll need to mix and match these. For example, the reference loop will often start off life as a range loop. Unpacking loopLet's start unpacking a string "by hand":<jupyter_code>myString = 'word' # define<jupyter_output><empty_output><jupyter_text>Print out each letter of this word:<jupyter_code>print(myString[0])
print(myString[1])
print(myString[2])
print(myString[3])<jupyter_output>w
o
r
d<jupyter_text>**Q:** What are some issues we can see with doing things this way?  Let's do this whole thing with a for-loop: I'm just going to write the cell, run it and then we'll talk about it:<jupyter_code>for letter in myString:
    print(letter)<jupyter_output>w
o
r
d<jupyter_text>We can see that there are some bold words in there, some of our special words: `for` and `in` and Jupyter notebook is trying to help us get that these are special words by making them bold+green.Let's look at the *syntax* (i.e. grammar/spelling) of what we are doing in this for-loop:  Core syntax referenceWhat does this for loop look like?```pythonfor iterable_variable in sequence:    do this stuff```This will loop over the sequence you provide, unpacking each value into the `iterable_variable`. This variable will hold an individual value, one at a time, for each loop through.Let's give names to all of these things: * declared and opened with a `for` keyword * a sequence that is being iterated over * an iterable variable to hold the variable contents of the sequence during each loop * white space (the tab) that defines where the executable code lives * the code to be executed each loop Some important notes: * the iterable variable name can be literally any valid variable name and does not need to be declared previously * the colon at the end of the for line block is required and indicates that the declaration line is done * the in keyword separates the iterable variable name from the sequence * this case has the sequence literal in the declaration line, but could also be a variable Let's take a moment to pause and look at this iterable variable more closely - note that I didn't define it anywhere but in the loop!  It can in fact be called anything we'd like, for example:<jupyter_code>for WHATEVER in myString:
    print(WHATEVER)<jupyter_output>w
o
r
d<jupyter_text>But note that after the loop the variable now exists and we can use it!<jupyter_code>print(WHATEVER)<jupyter_output>d<jupyter_text>The ways we usually make mistakes with for-loops are (**SHOW**): * indenting errors  * forgetting a colon * replacing the variable we are iterating over with something that can't be iterated over (try this with 1 vs [1]) What can we loop over?  We'll we just did strings, but we can also do lists:<jupyter_code>myList = [5, 6.6, 'bob']
for element in myList:
    print(element)<jupyter_output>5
6.6
bob<jupyter_text>We can even do nested for loops!<jupyter_code>myListOfStrings = ['bob', 'linda', 'tina', 'louise']
for string in myListOfStrings:
    #print(string) # a good first step for any new loop
    for letter in string:
        print(letter)<jupyter_output>b
o
b
l
i
n
d
a
t
i
n
a
l
o
u
i
s
e<jupyter_text>What else?  What if we just want to count something?  Let's try that:<jupyter_code>for num in range(5):
    print(num)<jupyter_output>0
1
2
3
4<jupyter_text>What is this range thing here?  Its basically a function that only "unwraps" during something like a for loop:<jupyter_code>range(7)<jupyter_output><empty_output><jupyter_text>We can kind of index it like a list:<jupyter_code>range(7)[0]
range(7)[1]
range(7)[5]<jupyter_output><empty_output><jupyter_text>We can even explicitly use `range` as a list if we turn it into one:<jupyter_code>print(list(range(11)))<jupyter_output>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]<jupyter_text>Let's use `range` to loop over a string:<jupyter_code>myString = 'word'

for num in range(4):
    print(myString[num])<jupyter_output>w
o
r
d<jupyter_text>Now, we could have unpacked this string in a loop like before, but this gives us an excuse to use the "len" function to loop through the length of any old string:<jupyter_code>myString = 'any old string!'
print(len(myString))
for num in range(len(myString)):
    print(myString[num])<jupyter_output>a
n
y
 
o
l
d
 
s
t
r
i
n
g
!<jupyter_text>Why bother with this "num" thing at all?  Well we can do things like print out where each character is:<jupyter_code>for num in range(len(myString)):
    print("The character at position", num, "is", myString[num])<jupyter_output>The character at position 0 is a
The character at position 1 is n
The character at position 2 is y
The character at position 3 is  
The character at position 4 is o
The character at position 5 is l
The character at position 6 is d
The character at position 7 is  
The character at position 8 is s
The character at position 9 is t
The character at position 10 is r
The character at position 11 is i
The character at position 12 is n
The character at position 13 is g
The character at position 14 is !<jupyter_text>if-then statementsBefore getting into making decisions in our code, we have to understand a bit about a new type of variable a `Boolean`.Generally speaking, booleans allow us to make decision statements. We can use this to check inputs, create more complex algorithms, and data filtering.Let's review an essential concepy: the expression. These are pieces of code that evaluate to a value. For example: * 2 + 2 evaluates to 4 * "hello there".split()" evaluates to a list ["hello", "there"]Sometimes we need to do more than just evaluate something. Sometimes you need to do something if something else is true. To do this, we need to introduce something new: decision statements.Crafting a decision statement requires that you create a boolean expression. To do this, we need two words: True or False.And before we continue, there are two things to consider. Booleans are both 1). a concept (logic, etc) and 2). a data type. * **boolean:** this is a data type, representing the two logical objects     * `True` and `False`: these are the two boolean objects available in Python, note that capitalization matters * **conditional check:** this is an expression will end up as `True` or `False`. Examples will be anything that has the == in it, or a method/function that returns a `True` or `False`. You can also use != to indicate 'not equal to'.  Crafting a boolean expressionThese are expressions that evaluate to either `True` or `False` and there are several ways to create them. The two primary ways are using an operator (math looking thing) or a method/function (including a custom function that you write!) specifically designed to return a boolean value. core boolean operators * == (read as "double equals") this is the equality check operator, which should appear between two objects and will return either True or False, e.g.:   * 1 == 1 will return True   * 2 == 4 will return False   * Important: this will check equivalence and not object identity. This is a concept that we will not really bo covering too heavily. But you may see the is keyword, which will check identity. This may appear to be functionally equivalent, but that is not the case. For our purposes, avoid the use of is.   Then the usual math suspects: * $>$ (greater than) * $<$ (less than) * $<=$ (less than or equal to) * $>=$ (greater than or equal to) There is an important distinction between the greater/less than and the greater/less than or equal to operators. Example: 10 is not less than 10, but it is equal to it. So if you were trying to check if something is a number under, but including 10, you need to use num <= 10. This is functionall equivalent to num < 11. You well develop your own personal style. Let's try evaluating some booleans.  For example, we know that 2+2=4:<jupyter_code>2+2<jupyter_output><empty_output><jupyter_text>(Note that I didn't include a `print` statement here -- you don't have to in jupyter notebooks if its the last line in the cell, but this is not true for other editors, so you've been warned!) Now, let's say we want to double check that 2+2 is 4... how could we do this?  With a Boolean op:<jupyter_code>2+2 == 4<jupyter_output><empty_output><jupyter_text>Because 2+2 does indeed equal 4, we get the following expression evaluated to True.  What about 2+3?  Does this equal 4?  Let's check:<jupyter_code>2+3 == 4<jupyter_output><empty_output><jupyter_text>Nope!  It does not!  We could even go further and check if 2+3 explicitly does not equal 4:<jupyter_code>2+3 != 4<jupyter_output><empty_output><jupyter_text>This expression evaluates to True because 2+3 indeed does not equal 4.  We can also test if 2+3 is less than 4:<jupyter_code>2+3 < 4<jupyter_output><empty_output><jupyter_text>Or greater than 4:<jupyter_code>2+3 > 4<jupyter_output><empty_output><jupyter_text>You can also do this with strings by the way, like for example, checking to see if a sub-string is in a string:<jupyter_code>'bow' in 'davidbowie'
'tie' in 'davidbowie'<jupyter_output><empty_output><jupyter_text>Now back to numbers.   Let's re-write this with variables:<jupyter_code>var1 = 2
var2 = 3

var1+var2 == 4<jupyter_output><empty_output><jupyter_text>Now, let's do something if our 2 variables equal 4 and something else if they do not.  We'll do this with an if-then statement:<jupyter_code>if var1+var2 == 4:
    print('the sum is 4!')<jupyter_output><empty_output><jupyter_text>So!  Nothing prints because var1+var2 = 5, which is not 4.  We probably want to make sure we print out this fact as well, just to be sure (we don't have to, the above if-then is totally fine, but we can extend it):<jupyter_code>if var1+var2 == 4:
    print('the sum is 4!')
else:
    print('the sum is NOT 4!')<jupyter_output>the sum is NOT 4!<jupyter_text>So, we can see that indeed 2+3 is not 4 and our if-then statement gets this! Let's try it again with 2+2:<jupyter_code>var1 = 2
var2 = 2

if var1+var2 == 4:
    print('the sum is 4!')
else:
    print('the sum is NOT 4!')<jupyter_output>the sum is 4!<jupyter_text>Cool!  It works!  There is a LOT more info here: https://github.com/elliewix/IS-452-Spring2020/blob/master/Lectures/Week-07-BooleansPt1-if-else.ipynbBut just to get a feel for it, let's try a few other ways of adding conditions with an `elif` or "else-if" statement:<jupyter_code>var1 = 2
var2 = 2

if var1+var2 == 4:
    print('the sum is 4!')
elif var1+var2 == 5:
    print('the sum is totally 5!')
else:
    print('the sum is NOT 4 OR 5!')<jupyter_output>the sum is 4!<jupyter_text>Let's try a few different numbers:<jupyter_code>var1 = 2
var2 = 3

if var1+var2 == 4:
    print('the sum is 4!')
elif var1+var2 == 5:
    print('the sum is totally 5!')
else:
    print('the sum is NOT 4 OR 5!')
var1 = -6
var2 = 11

if var1+var2 == 4:
    print('the sum is 4!')
elif var1+var2 == 5:
    print('the sum is totally 5!')
else:
    print('the sum is NOT 4 OR 5!')
var1 = -6
var2 = 1

if var1+var2 == 4:
    print('the sum is 4!')
elif var1+var2 == 5:
    print('the sum is totally 5!')
else:
    print('the sum is NOT 4 OR 5!')<jupyter_output>the sum is NOT 4 OR 5!<jupyter_text>Examples using `matplotlib` Now, we are going to use some of these ideas to (finally!) play around with a bit of plotting.  We'll do more of this tomorrow as well, but let's get started anyway!<jupyter_code>import matplotlib.pyplot as plt # this is shorthand you'll see alot too<jupyter_output><empty_output><jupyter_text>Let's make a simple plot of some random numbers:<jupyter_code>plt.plot([1,2,3])<jupyter_output><empty_output><jupyter_text>Not too exciting.  Note it tells us a bit about where this plot is stored in memory.  If we want to just show the plot we can add another line:<jupyter_code>plt.plot([1,2,3])
plt.show() # show, not show and tell!<jupyter_output><empty_output><jupyter_text>From this plot its a bit hard to tell where our data points are.  Also, we've only given it 3 numbers, how does it know how to plot x and y?  Let's figure out where our points are and maybe that will help.  One way we can do this is by messing with the plot stylings:<jupyter_code>plt.plot([1,2,3], 'o')
plt.show()<jupyter_output><empty_output><jupyter_text>Now, we've said hey, instead of plotting some default line, please plot some points instead!   So we see that by default, we are plotting the list values by their indicies.There are actually a HUGE number of ways to format our plot and they are listed here: https://matplotlib.org/api/_as_gen/matplotlib.pyplot.plot.htmlmatplotlib.pyplot.plotLet's just focus on a few for right now.  First, let's start by plotting an actual x vs. y:<jupyter_code>x = [5,6,7]
y = [5.5, 8.8, 4.4]

plt.plot(x,y, 'o')
plt.show()<jupyter_output><empty_output><jupyter_text>Let's plot a line connecting these points too:<jupyter_code>x = [5,6,7]
y = [5.5, 8.8, 4.4]

plt.plot(x,y, 'o-')
plt.show()<jupyter_output><empty_output><jupyter_text>We can also mess with the color of the line:<jupyter_code>x = [5,6,7]
y = [5.5, 8.8, 4.4]

plt.plot(x,y, 'o-', color='red')
plt.show()<jupyter_output><empty_output><jupyter_text>Note that sometimes you'll see a little short hand for this like:<jupyter_code>x = [5,6,7]
y = [5.5, 8.8, 4.4]

plt.plot(x,y, 'ro-')
plt.show()<jupyter_output><empty_output><jupyter_text>But we are going to try to keep it explicit color-wise! We can also plot multiple lines on the same plot:<jupyter_code>x1 = [5,6,7]
y1 = [5.5, 8.8, 4.4]

x2 = [4, 5, 7]
y2 = [6.5, 9, 4.3]

plt.plot(x1,y1)
plt.plot(x2, y2)
plt.show()<jupyter_output><empty_output><jupyter_text>Note that if I don't specify any colors there are defaults chosen for me, but we can also specify them explicitly like we did before:<jupyter_code>x1 = [5,6,7]
y1 = [5.5, 8.8, 4.4]

x2 = [4, 5, 7]
y2 = [6.5, 9, 4.3]

plt.plot(x1,y1, 'o-', color='red')
plt.plot(x2, y2, 's-', color='blue') # s stands for square!
plt.show()<jupyter_output><empty_output><jupyter_text>Let's make our plots fancier -- first, let's label the x & y axis:<jupyter_code>x1 = [5,6,7]
y1 = [5.5, 8.8, 4.4]

x2 = [4, 5, 7]
y2 = [6.5, 9, 4.3]

plt.plot(x1,y1, 'o-', color='red')
plt.plot(x2, y2, 's-', color='blue') # s stands for square!

plt.xlabel('x')
plt.ylabel('y')

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09164101349751362, 'fraction_numerical': 0.023679848448969927, 'mean_word_length': 4.789239204934887, 'pattern_counts': {'":': 1, '<': 163, '<?xml version=': 0, '>': 161, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '44', 'dir': 'jupyter-structured-clean-dedup', 'id': '86321', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 5775, 'n_tokens_neox': 5450, 'n_words': 2390, 'path': 'lesson02/.ipynb_checkpoints/prep_notebook_lesson02-checkpoint.ipynb', 'repo_name': 'jnaiman/csci-p-14110_su2020', 'size': '16892'}