<jupyter_start><jupyter_text>Ex2 - Getting and Knowing your Data This time we are going to pull data directly from the internet.Special thanks to: https://github.com/justmarkham for sharing the dataset and materials. Step 1. Import the necessary libraries<jupyter_code>import pandas as pd<jupyter_output><empty_output><jupyter_text>Step 2. Import the dataset from this [address](https://raw.githubusercontent.com/justmarkham/DAT8/master/data/chipotle.tsv).   Step 3. Assign it to a variable called chipo.<jupyter_code>chipo = pd.read_csv('https://raw.githubusercontent.com/justmarkham/DAT8/master/data/chipotle.tsv', sep='\t')<jupyter_output><empty_output><jupyter_text>Step 4. See the first 10 entries<jupyter_code>chipo.head(10)<jupyter_output><empty_output><jupyter_text>Step 5. What is the number of observations in the dataset?<jupyter_code># Solution 1
chipo.shape[0]
# Solution 2
chipo.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 4622 entries, 0 to 4621
Data columns (total 5 columns):
order_id              4622 non-null int64
quantity              4622 non-null int64
item_name             4622 non-null object
choice_description    3376 non-null object
item_price            4622 non-null object
dtypes: int64(2), object(3)
memory usage: 180.7+ KB<jupyter_text>Step 6. What is the number of columns in the dataset?<jupyter_code>print(chipo.columns.size)
print(chipo.shape[1])<jupyter_output>5
5<jupyter_text>Step 7. Print the name of all the columns.<jupyter_code>chipo.columns<jupyter_output><empty_output><jupyter_text>Step 8. How is the dataset indexed?<jupyter_code>chipo.index<jupyter_output><empty_output><jupyter_text>Step 9. Which was the most-ordered item?<jupyter_code>chipo.groupby('item_name').sum().sort_values(by='quantity', ascending=False).head(1)
chipo.iloc[chipo['quantity'].idxmax()]['item_name'] #most in one order<jupyter_output><empty_output><jupyter_text>Step 10. For the most-ordered item, how many items were ordered?<jupyter_code>chipo.groupby('item_name').sum().sort_values(by='quantity', ascending=False).head(1)['quantity']
chipo.iloc[chipo['quantity'].idxmax()]['quantity'] #most in one order<jupyter_output><empty_output><jupyter_text>Step 11. What was the most ordered item in the choice_description column?<jupyter_code>chipo.groupby('choice_description').sum().sort_values(by='quantity', ascending=False).head(1)['quantity']
#if most in one order with choice_description != np.nan
chipo_nonans = chipo.dropna().copy()
chipo_nonans.reset_index(inplace=True)
chipo_nonans.iloc[chipo_nonans['quantity'].idxmax(axis=1)]<jupyter_output><empty_output><jupyter_text>Step 12. How many items were orderd in total?<jupyter_code>chipo['quantity'].sum()<jupyter_output><empty_output><jupyter_text>Step 13. Turn the item price into a float  Step 13.a. Check the item price type<jupyter_code>chipo['item_price'].dtype.name<jupyter_output><empty_output><jupyter_text>Step 13.b. Create a lambda function and change the type of item price<jupyter_code>#Method 1: Self-defined lambda using apply
chipo['item_price'] = chipo['item_price'].apply(lambda r: float(r.replace('$', '')))

#.apply(lambda r: float(r[1:-1])) also acceptable if it's known that the dollar sign position is fixed (and it's likely faster)
# Note: the following methods don't work because pandas doesn't know how to parse a numerical string that has a $ sign in it.

# chipo['item_price'].infer_objects().dtypes.name
# >>> object

#Method 2A: to_numeric() in apply()
# chipo['item_price'].apply(pd.to_numeric)

#Method 2B: to_numeric() alone
# pd.to_numeric(chipo['item_price'], errors='raise', downcast='float')

#Method 3: astype()
# chipo['item_price'].astype('float64', errors='raise')<jupyter_output><empty_output><jupyter_text>Step 13.c. Check the item price type<jupyter_code>chipo['item_price'].dtype.name<jupyter_output><empty_output><jupyter_text>Step 14. How much was the revenue for the period in the dataset?<jupyter_code>chipo['revenue'] = chipo['quantity']*chipo['item_price']
display(chipo['revenue'])
total_revenue = chipo['revenue'].sum()
print(total_revenue)<jupyter_output><empty_output><jupyter_text>Step 15. How many orders were made in the period?<jupyter_code>chipo.groupby('order_id').count().index.size<jupyter_output><empty_output><jupyter_text>Step 16. What is the average revenue amount per order?<jupyter_code># Solution 1
chipo.groupby('order_id').sum().mean()['revenue']<jupyter_output><empty_output><jupyter_text>Step 17. How many different items are sold?<jupyter_code>chipo['item_name'].value_counts()
#alternatively
chipo['item_name'].value_counts().index.size<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12989247311827956, 'fraction_numerical': 0.02129032258064516, 'mean_word_length': 7.3953068592057765, 'pattern_counts': {'":': 0, '<': 72, '<?xml version=': 0, '>': 75, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '570492', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1678, 'n_tokens_neox': 1583, 'n_words': 351, 'path': '01_Getting_&_Knowing_Your_Data/Chipotle/Exercises.ipynb', 'repo_name': 'slcheungcasado/pandas_exercises', 'size': '4650'}