<jupyter_start><jupyter_text>Agenda  1.  [Review](0)2.  [Pandas](2)2.  [Exercise](10) 3.  [Exercise](12)      Review<jupyter_code><jupyter_output><empty_output><jupyter_text>ExerciseCreate a horizontal bar chart using the data below.  Use an 'X'data = { 'New York':20,     'California':10,     'Rhode Island':5,     'Texas': 15,     'California':17,     'Florida':8,     'New Hampshire':3,     'New Jersey':12}Do not use matplotlib. Hint: dataframes can use df.plot(kind = 'chart_type'). Try this out.  Solution<jupyter_code>import pandas as pd
data = { 'New York':20, 
    'California':10, 
    'Rhode Island':5, 
    'Texas': 15, 
    'California':17, 
    'Florida':8, 
    'New Hampshire':3, 
    'New Jersey':12}
x = pd.DataFrame(data = data.values(), index = data.keys())
x.plot(kind = 'barh')<jupyter_output><empty_output><jupyter_text>Pandas DataFramesWe can add an index to our data frame by using the index parameter<jupyter_code>import numpy as np
import pandas as pd<jupyter_output><empty_output><jupyter_text>ExerciseUsing a pandas dataframe use the dictionary below and transfer the states to the index.data = { 'New York':20,     'California':10,     'Rhode Island':5,     'Texas': 15,     'California':17,     'Florida':8,     'New Hampshire':3,     'New Jersey':12} Print out the results of California to Florida Solution<jupyter_code>x['California':'Florida']<jupyter_output><empty_output><jupyter_text>Remember that the loc method is inclusive!!!! Pandas - Add column headersTo add a column header use the columns parameter.<jupyter_code>import pandas as pd
data = { 'New York':[20,15], 
    'California':[10,3], 
    'Rhode Island':[5,6], 
    'Texas': [15,3], 
    'California':[17,4], 
    'Florida':[8,9], 
    'New Hampshire':[3,4], 
    'New Jersey':[12,8]}<jupyter_output><empty_output><jupyter_text>add a new column, 'votes', which is the total of the Yes and No Vote of a referendum on tax control by state:import pandas as pddata = { 'New York':[20,15],     'California':[10,3],     'Rhode Island':[5,6],     'Texas': [15,3],     'California':[17,4],     'Florida':[8,9],     'New Hampshire':[3,4],     'New Jersey':[12,8]}<jupyter_code>import pandas as pd
data = { 'New York':[20,15], 
    'California':[10,3], 
    'Rhode Island':[5,6], 
    'Texas': [15,3], 
    'California':[17,4], 
    'Florida':[8,9], 
    'New Hampshire':[3,4], 
    'New Jersey':[12,8]}
x['Votes'] = data.values()
x<jupyter_output><empty_output><jupyter_text>Pandas - Remove a columnThere are 2 ways to remove a column from a dataframe.Method 1:To drop a column use the drop method with the axis parameterimport pandas as pddata = { 'New York':[20,15],     'California':[10,3],     'Rhode Island':[5,6],     'Texas': [15,3],     'California':[17,4],     'Florida':[8,9],     'New Hampshire':[3,4],     'New Jersey':[12,8]}NOTE: The drop is not executed in place.  You need to rename the dataframe to see    your changes<jupyter_code>y = x.drop(['Votes'], axis=1)
y<jupyter_output><empty_output><jupyter_text>Pandas - Remove a column - Method 2Use the pop method to remove the column in placeimport pandas as pddata = { 'New York':[20,15],     'California':[10,3],     'Rhode Island':[5,6],     'Texas': [15,3],     'California':[17,4],     'Florida':[8,9],     'New Hampshire':[3,4],     'New Jersey':[12,8]}<jupyter_code>import pandas as pd
data = { 'New York':[20,15], 
    'California':[10,3], 
    'Rhode Island':[5,6], 
    'Texas': [15,3], 
    'California':[17,4], 
    'Florida':[8,9], 
    'New Hampshire':[3,4], 
    'New Jersey':[12,8]}
#x = pd.DataFrame(data = data.values(), keys = data.keys())
y = x.pop('Votes')<jupyter_output><empty_output><jupyter_text>ExerciseTake the columns yes and no ([yes,no]) and transform them from values to a percentage of the votes for each Stateimport pandas as pddata = { 'New York':[20,15],     'California':[10,3],     'Rhode Island':[5,6],     'Texas': [15,3],     'California':[17,4],     'Florida':[8,9],     'New Hampshire':[3,4],     'New Jersey':[12,8]} Solution<jupyter_code>import pandas as pd
data = { 'New York':[20,15], 
    'California':[10,3], 
    'Rhode Island':[5,6], 
    'Texas': [15,3], 
    'California':[17,4], 
    'Florida':[8,9], 
    'New Hampshire':[3,4], 
    'New Jersey':[12,8]}
x = pd.DataFrame(data =data.values(), index = data.keys())
x['Yes Votes'] = x[0]/(x[1]+x[0])*100
x['No Votes'] = x[1]/(x[0]+x[1])*100
x.pop(0)
x.pop(1)
x<jupyter_output><empty_output><jupyter_text>Exercise:Using our US_Presidents.txt file and Pandas1) Clean the data2) Reorder all the presidents by their index. Which is the number of president they are. Washington is 1st, Adams is 2nd, etc3) Create columns first name, middle name, last name. If they don't have a a middle name, leave it blank4) Add a column called term length. Which tells us how long they were in office for<jupyter_code>import pandas as pd 
import numpy as np
pres = pd.read_csv('US_Presidents_unordered.txt')
pres = pd.DataFrame(pres)
#1 Clean the data - we recognize unwanted quotes 
pres.columns = ['Name', 'Term Start', 'Term End']
pres['Name'] = pres['Name'].map(lambda x: x.strip("'"))
pres["Term Start"] = pres["Term Start"].map(lambda x: x.strip("'"))
pres["Term End"] = pres["Term End"].map(lambda x: x.strip("'"))

#2 Reorder by index - we create a new column to store the index
pres['Pres_ind'] = pres['Name'].map(lambda x: x[0:2].strip(".")).astype(int)
pres = pres.sort_values('Pres_ind')
pres

#3 Columns for first, middle, last name 


#4 Term Length 
pres['Term Start'] = pd.to_datetime(pres['Term Start'])
pres['Term End'] = pd.to_datetime(pres['Term End'])
pres['Term Length'] =  pres['Term End'].subtract(pres['Term Start'])
pres['Term Length'] = pres['Term Length']/ np.timedelta64(1, 'Y')
pres<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17497403946002077, 'fraction_numerical': 0.0446521287642783, 'mean_word_length': 4.546065259117083, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '470440', 'license': 'BSD-Source-Code', 'n_tokens_mistral': 2242, 'n_tokens_neox': 2034, 'n_words': 558, 'path': 'CI_Data_Science_Lesson_12_with_ANSWERS.ipynb', 'repo_name': 'MaxDGU/datasciencenotebooks', 'size': '5778'}