<jupyter_start><jupyter_text>Ex2 - Getting and Knowing your Data This time we are going to pull data directly from the internet.Special thanks to: https://github.com/justmarkham for sharing the dataset and materials. Step 1. Import the necessary libraries<jupyter_code>import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>Step 2. Import the dataset from this [address](https://raw.githubusercontent.com/justmarkham/DAT8/master/data/chipotle.tsv).   Step 3. Assign it to a variable called chipo.<jupyter_code>chipo = pd.read_csv('chipotle.tsv', sep = "\t", header = 0) #read_csv reads coma separated value files, but bc this was a tsv, you need to specify that it is separated by tabs
chipo<jupyter_output><empty_output><jupyter_text>Step 4. See the first 10 entries<jupyter_code>chipo.head(10) #head shows the first - entries, tail shows the last - entries
chipo.iloc[:10] #this looks for the 10th row
chipo.loc[:10] #this looks for the row named 10
chipo.index = ["row_" + str(i) for i in chipo.index]
chipo.head(10)
chipo.iloc[:10]
chipo.loc[:'row_10', 'item_price']<jupyter_output><empty_output><jupyter_text>Step 5. What is the number of observations in the dataset?<jupyter_code># Solution 1
len(chipo)

# Solution 2
chipo.shape<jupyter_output><empty_output><jupyter_text>Step 6. What is the number of columns in the dataset?<jupyter_code>len(chipo.columns) #columns lists all the columns in an array, so finding the length of this gives the number of columns<jupyter_output><empty_output><jupyter_text>Step 7. Print the name of all the columns.<jupyter_code>for col in chipo.columns: print (col) #in python loops are easier
n = len(chipo.columns) #more traditional method

i = 0

while i < n:
    col = chipo.columns[i]
    print(col)
    i += 1<jupyter_output>order_id
quantity
item_name
choice_description
item_price<jupyter_text>Step 8. How is the dataset indexed?<jupyter_code>chipo.index<jupyter_output><empty_output><jupyter_text>Step 9. Which was the most-ordered item?<jupyter_code>most_ordered = chipo.groupby('item_name').agg({'quantity':['sum', "max"]})
most_ordered.columns #.sort_values('quantity', ascending = False).head(10)
chipo.groupby('item_name')[['quantity']].sum().sort_values('quantity', ascending = False).head(10)<jupyter_output><empty_output><jupyter_text>Step 10. For the most-ordered item, how many items were ordered?<jupyter_code>#most_ordered.sort_values('quantity', ascending = False).head(1)<jupyter_output><empty_output><jupyter_text>Step 11. What was the most ordered item in the choice_description column?<jupyter_code>most_ordered = chipo.groupby('choice_description').agg({'quantity':'sum'})
most_ordered.sort_values('quantity', ascending = False).head(1)<jupyter_output><empty_output><jupyter_text>Step 12. How many items were orderd in total?<jupyter_code>chipo.quantity.sum()<jupyter_output><empty_output><jupyter_text>Step 13. Turn the item price into a float  Step 13.a. Check the item price type<jupyter_code>price = chipo.item_price
price
price.iloc[0]
def f(price_val):
    return price_val[1:-1]
x = price.iloc[0]
f(x)
f(price.loc['row_4619'])
f = lambda price_val: float(price_val[1:-1])
f(price.loc['row_4619'])<jupyter_output><empty_output><jupyter_text>Step 13.b. Create a lambda function and change the type of item price<jupyter_code>chipo['item_price_mapped'] = chipo.item_price.map(f)
chipo
type(chipo.loc['row_0','item_price'])
def f():
    print('Hello World!')
f()
f = lambda : print('Hello World!')
f()
(lambda : print('Hello World!'))()<jupyter_output>Hello World!<jupyter_text>Step 13.c. Check the item price type<jupyter_code>chipo.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Index: 4622 entries, row_0 to row_4621
Data columns (total 5 columns):
 #   Column              Non-Null Count  Dtype  
---  ------              --------------  -----  
 0   order_id            4622 non-null   int64  
 1   quantity            4622 non-null   int64  
 2   item_name           4622 non-null   object 
 3   choice_description  3376 non-null   object 
 4   item_price          4622 non-null   float64
dtypes: float64(1), int64(2), object(2)
memory usage: 345.7+ KB<jupyter_text>Step 14. How much was the revenue for the period in the dataset?<jupyter_code>chipo.item_price.sum()<jupyter_output><empty_output><jupyter_text>Step 15. How many orders were made in the period?<jupyter_code>chipo.quantity.sum()<jupyter_output><empty_output><jupyter_text>Step 16. What is the average revenue amount per order?<jupyter_code># Solution 1
chipo.item_price.sum() / len(chipo)

# Solution 2
chipo.item_price.mean()<jupyter_output><empty_output><jupyter_text>Step 17. How many different items are sold?<jupyter_code>chipo.item_name.nunique()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11972274732199117, 'fraction_numerical': 0.027515227893299726, 'mean_word_length': 5.487738419618529, 'pattern_counts': {'":': 0, '<': 72, '<?xml version=': 0, '>': 71, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '273225', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1755, 'n_tokens_neox': 1640, 'n_words': 423, 'path': '01_Getting_&_Knowing_Your_Data/Chipotle/Exercises.ipynb', 'repo_name': 'abelle4577/pandas_exercises', 'size': '4761'}