<jupyter_start><jupyter_text>Lab 8: Python Basics and Visualization In this lab you will become more familiarized with Python.The goal of this lab is that you learn:  1. **Variables and basic data types in Python**  2. **Visualization in Python**  3. **Use of conditionals**  4. **Use of for-loops**        1. Python's Variables and Data Types:- Numerals- Variables- Strings- Lists- Arrays  Numerals Python uses float (numbers with decomal points) and int (integers) as numerals.  With numerals, Python can be used as a calculator. Run the cells below to see it for yourself.<jupyter_code>3+5
140-20
40/2
2*2<jupyter_output><empty_output><jupyter_text>Python follows the same arithmetic rules as we do. So guess what the output of the next cell will be:<jupyter_code>2*5+1<jupyter_output><empty_output><jupyter_text>What about this cell:<jupyter_code>2*(5+1)<jupyter_output><empty_output><jupyter_text>YOUR TURN:  Guess what a double star notation does? Run the cell below to see if you had the right intuition about it.<jupyter_code>3**2<jupyter_output><empty_output><jupyter_text>As was mentioned above, there are two types of numerals: integers and floats. In older versions of Python, they were not compatible (one couldn't have performed any calculations on them without making them the same data type) If you must use older versions of Python for your work, it isimportant to remember which type you are using in order to avoid bugs. Floats use decimal points, so it will be easy for you to spot them. See how the output changes when we print the same number as an integer and as a float.<jupyter_code>int(1776)
float(1776)<jupyter_output><empty_output><jupyter_text>Variables To store the values in Python (and in many other programming languages) we need to use variables. Variables can store all kinds of information. This is useful if you plan to use the same values later on in your project. It will save you time and make your code more readable. To see how we store values in variables run the cell below.<jupyter_code>x=3
print(x)
burr = "Wait for it!"
burr<jupyter_output><empty_output><jupyter_text>If you don't assign any value to the variable and call it later, it will give you an error. We can see it in the cell below. (It will give you an error, but it was planned that way)<jupyter_code>burr
#y
y = 8
z = x+y
z<jupyter_output><empty_output><jupyter_text>Strings The phrases and words in Python are always stored inbetween double or single quotation marks. They are formally called strings.<jupyter_code>"History has its eyes on you"<jupyter_output><empty_output><jupyter_text>Also, you can use both single and double quotes for the strings, but sometimes you need to use single quotes inside of the string and that might cause a problem (it will "exit it out"). You can just always use the double quotation marks on the outside to avoid such problem. Or, you can use a backslash before the single quotation mark. It will tell Python to ignore it and not to exit it out. You can see an example of it below:<jupyter_code>print("What'd I miss")
print('You\'ll be back')<jupyter_output><empty_output><jupyter_text>The strings can also be "added" onto one another. Run the cell below to see how it works.<jupyter_code>"York"+"town"
"Mexico"+" "+"City"<jupyter_output><empty_output><jupyter_text>When we are using Jupyter Notebooks, the last line is printed out automatically (unless it is performing some other operation, eg. getting assigned to a specific value). But if you would like to print more than just the last line, you can use the print( ) method. If you just run the next cell, it will not give you any output.<jupyter_code>aaron = "Ev’ryone give it up for America’s favorite fighting Frenchman"
everyone = "Lafayette!"
mdl = "I’m takin this horse by the reins"<jupyter_output><empty_output><jupyter_text>YOUR TURN:  Try using print( ) in the cell below to output all the three lines of lyrics of "Guns and Ships".       Hint:  you'll need to use it three times.<jupyter_code>### YOUR CODE HERE ###<jupyter_output><empty_output><jupyter_text>A string is a sequence of characters. If you want to see how many characters your string has, use the len() method. Do you think spaces are considered characters in Python? Check if your intuition was correct by running the cell below.<jupyter_code>burr
len(burr)<jupyter_output><empty_output><jupyter_text>YOUR TURN:  Now create a variable name in which you will save your full name. Note that you need to call the variable once again in order to get an output.<jupyter_code>### YOUR CODE HERE ###<jupyter_output><empty_output><jupyter_text>Sometimes you need to switch the values in two variables. In the cell below we have the last names mixed up. The variable theodosia should have an output "burr" and philip should return "hamilton".How would you approach this problem without manually retyping the outputs?**Hint** Use a temporary variable in which you can temporarily save the value of one of the other variables.<jupyter_code>theodosia = "hamilton"
philip = "burr"
print(theodosia)
print(philip)
##fill these in, Version 1:
theodosia, philip = "burr", "hamilton"
print(theodosia)
print(philip)
theodosia = "hamilton"
philip = "burr"
# Version 2
temp = theodosia
theodosia = philip
philip = temp

print(theodosia)
print(philip)<jupyter_output><empty_output><jupyter_text>Lists You have learned about the strings and numerals, it's time to take it up a notch. Another useful and very popular data type is a Python list.Lists can take various data types (both numerals, strings, arrays, other lists, etc). To create an empty list just make your variable equal to "[ ]", like so:<jupyter_code>washington = []
washington<jupyter_output><empty_output><jupyter_text>To add the values into your list you can either use lst.append( ), lst.insert( ), or lst.extend( ).<jupyter_code>washington.extend(["Right Hand Man"])
washington
washington.insert(0, "I need my")
washington
washington.append("Back")
washington<jupyter_output><empty_output><jupyter_text>Notice, that the `insert` method takes two positional values. You need to specify the index in the list at which it will be inserted, whilst `append` just adds the value to the end of the list. `extend` takes another list as its input. If you try to use a string or a number by itself, it will give you an error. When you need to access an element at a specific index, that is the code you can use The code below returns the second element of the list.<jupyter_code>washington[1]<jupyter_output><empty_output><jupyter_text>**Note:** Python starts counting at 0, not 1. Sometimes you just need to use part of the list. In this case you can use ":" to modify your list. Let's make a new list: Sometimes you just need to use part of the list or copy the whole list into a new variable. In thiscase you can use ":" to modify your list.The notation is:**a = b[start:stop:step]*****start***: the index you want to start with. (the default is 0)***stop***: the index you want to end with. (the default is the last number)***step***: use it, if you wish to skip some indeces (eg. use only every other value). For example, if you start with index 1 and step is 2, you will iterate through the 1st index, then 3rd, then 5th, etc. until you get to the stopping index. (the default is 1)Note that python will stop iterating at the value before the last index. See the following example:<jupyter_code>lst = [1,2,3,4,5]
lst
lst2=list(range(1,6))
lst2
lst[0:5:1]
lst3=list(range(1,21))
lst3
lst3[1:20:2]<jupyter_output><empty_output><jupyter_text>Note: if you wish to start iterating from the end of a list backwards, we use a negative step:<jupyter_code>lst[::-1]
ham_songs = ["<NAME>", "<NAME>", "My Shot", \
             "The Story of Tonight", "The Schuyler Sisters", "Farmer Refuted",
             "You'll be Back", "Right Hand Man", "A Winter's Ball", \
             "Helpless", "Satisfied"]

every_other = ham_songs[::2] 
every_other
lyrics = ["I", "am", "not", "throwing", "away", "my", "shot"]
lyrics<jupyter_output><empty_output><jupyter_text>Let's say you want to save only the two last words from your old list into your new list. That's what you will need to do then:<jupyter_code>alex = lyrics[-2:]
alex

# alternatively you can use lyrics[5:]<jupyter_output><empty_output><jupyter_text>Notice also that if you want to learn the length of your list, you can use the same method "len( )", but it will not count all the characters anymore. Instead, it will count all the elements in it. Like so:<jupyter_code>washington
len(washington)<jupyter_output><empty_output><jupyter_text>YOUR TURN:  Create a list with the values 1, 2, 3, 4, and 5 and only get the values 2 and 4<jupyter_code>### YOUR CODE HERE ###<jupyter_output><empty_output><jupyter_text>Arrays<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>With the help of numpy we can introduce a new data type - arrays. Arrays are commonly used with Data Frames (basically, tables of values). There are a lot of functions and methods that can be used with arrays when we are analyzing data. The main difference between the lists and arrays is that an array can take only one type of data (eg. only numerals, or strings, but never both).To create an array with values, just make your variable equal to np.array("your value")<jupyter_code>arr = np.array(10)
arr<jupyter_output><empty_output><jupyter_text>A very useful method that can be used with an array of numbers is np.arange( ). It takes at least 2, sometimes 3 positional arguments. The first number will be the starting point of your array, the last one will identify up to which number your array will go (not inclusive). The third positional argument is optional, it shows how many steps an array should skip through. (This is very similar to lists)For example, if you need an array with numbers from 0 to 20, but you only want for it to include every other number, you will need add a third positional argument.<jupyter_code>every_other = np.arange(0, 21, 2)
every_other<jupyter_output><empty_output><jupyter_text>You can also convert a list of numbers in Python into an array:<jupyter_code>num_lst = [1, 2, 3, 4, 5]
num_arr = np.array(num_lst)
num_arr<jupyter_output><empty_output><jupyter_text>You can do some arithmetic with arrays. Guess what will be the output of this line of code before you run it:<jupyter_code>num_arr*2<jupyter_output><empty_output><jupyter_text>As you can see, all the values in our array got mutliplied by 2. Try performing the same operation with the initial list of numbers instead. What do you think the output will be in the cell below?<jupyter_code>num_lst*2<jupyter_output><empty_output><jupyter_text>As you can see, lists and arrays not only can be used differently, but they also give different outputs when we are using the same operations on them.  **How to know what data type are we talking about?**<jupyter_code>lyrics
# Using type() and inside the parentheses including the object
type(lyrics)
"CP101"
type("CP101")
type(5)
type(5.4)<jupyter_output><empty_output><jupyter_text>2. Visualizations - introducing matplotlib! [Click here to check out the types of visualization you can output using matplotlib!](https://matplotlib.org/gallery/index.html) It is time for us to revisit Python libraries. What makes it powerful are the libraries people wrote for it. Libraries are the sets of different functions that you can import and use. The most popular libraries are numpy, pandas, and matplotlib.Since we are using Anaconda, a lot of libraries have been preloaded for you. You just need to "call" them. Just run the cell below. It will make our notebook know that we will be using these preinstalled libraries from that point on. Libraries are used to make Python more efficient. By itself (solely with built-in functions and methods), Python is not a very useful language, hence people keep building libraries (sets of functions and methods) that can be used together with Python in order to make it more useful.  Pandas - read in data<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt
# First let's upload the dataset we have selected to work on, using the pd.read_csv command.
broadway = pd.read_csv("data/broadway.csv")
print(type(broadway))
broadway.head()
broadway.tail(10)<jupyter_output><empty_output><jupyter_text>Sometimes it is easier to see a certain trend in our data by simply plotting the values from a few columns together. Like so:<jupyter_code>plt.scatter(broadway['Year'], broadway['Gross'])
plt.show()<jupyter_output><empty_output><jupyter_text>So we can see that the Gross amount for the shows were growing in the course of years. Which makes perfect sense. What if we wanted to see the same trend for a specific show? How about we see the changes in gross amount by week for all the years "The Phantom of the Opera" was on stage?<jupyter_code>phantom = broadway[broadway['Name'] == "The Phantom Of The Opera"]
phantom
plt.scatter(phantom['Year'], phantom['Gross'])
plt.show()<jupyter_output><empty_output><jupyter_text>YOUR TURN:  Use the same code for any other show you are interested in.<jupyter_code>### YOUR CODE HERE ###<jupyter_output><empty_output><jupyter_text>3. Conditionals Conditionals are also known as **booleans**. With conditional statements we can let the computer know when (aka under which condition) we want a specific operation to be executed. Under the hood the computer evaluates if the condition is True and performs a specific operation (like print( ), return( ), etc). Some of the most popular conditional statements are: **A == B: True if A equals B****A != B: True if A is not equal to B****A > B: True if A is greater than B. Same syntax for "<"****A >= B: True if A is greater than or equal to B. Same syntax for "<="** We have three types of statements in the Python conditionals: if, elif, and else. We always start with "if", we always end with "else". The "elif" statement is optional. It literally means "else if". While both "if" and "else" can only be used once, "elif" can be used multiple times, and let you add many more conditions to your statement.If the condition in the if case is False, then we move to the elif case. If the elif case is false, then we will move to the else case and perform whatever is inside the else condition. We move into the condition that has a True conditional value first and evaluate that.<jupyter_code>playwright = "<NAME>"

if playwright == "<NAME>":
    print (True)
else:
    print (False)<jupyter_output><empty_output><jupyter_text>**Note:** "=" and "==" are not the same operations in Python. A single equals sign (=) assignes the value to the right of it to the variable name to the left of it. While the double equals sign (==) compares if the value on the right is equal to the value to the left of it.  In the next line, let's ask Python to print the name of the school two of the Founding Fathers went based on their name. Feel free to change the string our "name" variable is assigned to.<jupyter_code>name = "Hamilton"

# Feel free to change the value of the variable "name"

if name == "Hamilton":
    print("Columbia")
elif name == "Burr":
    print("Princeton")
else:
    print("I don't know")<jupyter_output><empty_output><jupyter_text>In the cell below guess what the output will be, before you run it. Feel free to change the numeral under the variable "year".<jupyter_code>year = 1789

# Who's the president

if year>=1789 and year<1797:
    print("<NAME>")
elif year>=1797:
    print("<NAME>")
else:
    print("<NAME>")<jupyter_output><empty_output><jupyter_text>We can also search through lists of values in our conditional statements to see if a word is in our list using the keyword 'in':<jupyter_code>founding_fathers = ["Jefferson", "Hancock", "Hamilton", "Adams", "Washington", "Burr", "Madison"]

# change the name in "any_name"
if "any_name" in founding_fathers:
    print("You are not throwing away your shot!")
else:
    print("You've got to be in the room where it happens!")<jupyter_output><empty_output><jupyter_text>4. For-loops Loops (aka for-loops) allow for some code to be executed repeatedly. For example, if you wish to print the numbers from 0 to 10, you can do that with the for-loop. A for-loop iterates through a sequence of elements (list, string, array, etc.) and reassigns an element (which can have any name, most common are "x", "i", "elem", the name in itself doesn't matter) to each element of the sequence sequentially.<jupyter_code>for elem in ['a', 'b', 'c']:   
    print(elem)

elem<jupyter_output><empty_output><jupyter_text>That is why in the previous code cell we can see that when an "elem" is called at the end of the loop, it is equal to "c" which is the last element of the sequence. In the cell below we will be using the built-in function called range( ). It enumerates the numbers from 0 up to a number you put in (exclusive of that last number).**Note:** remember that Python starts to count from 0, hence the last number you specify is not going to be included.<jupyter_code>for i in range(11):
    print(i)<jupyter_output><empty_output><jupyter_text>YOUR TURN:  Let's combine what you know so far about lists, conditionals, and for loops:     Create a list with values 1, 2, 3, 4, 5. Iterate through the list with a for loop, and print only the values that are greater than 2.   Otherwise, print the statement "This value is not greater than 2."<jupyter_code>### YOUR CODE HERE ###<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08020350997541874, 'fraction_numerical': 0.008289029897673356, 'mean_word_length': 5.078526754690757, 'pattern_counts': {'":': 5, '<': 175, '<?xml version=': 0, '>': 176, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '41', 'dir': 'jupyter-structured-clean-dedup', 'id': '288626', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 5241, 'n_tokens_neox': 5038, 'n_words': 2489, 'path': 'Lab8/Lab 8 - Python Basics and Visualization.ipynb', 'repo_name': 'ds-modules/CP-101-SP21', 'size': '17558'}