<jupyter_start><jupyter_text>Ex3 - Getting and Knowing your Data This time we are going to pull data directly from the internet.Special thanks to: https://github.com/justmarkham for sharing the dataset and materials. Step 1. Import the necessary libraries<jupyter_code>import numpy as np
import pandas as pd
print("Pandas Version: " + pd.__version__)
print("Numpy Version: " + np.__version__)<jupyter_output>Pandas Version: 0.24.2
Numpy Version: 1.16.4<jupyter_text>Step 2. Import the dataset from this [address](https://raw.githubusercontent.com/justmarkham/DAT8/master/data/u.user).   Step 3. Assign it to a variable called users and use the 'user_id' as index<jupyter_code>users = pd.read_csv("https://raw.githubusercontent.com/justmarkham/DAT8/master/data/u.user", sep="|", 
                    index_col="user_id")<jupyter_output><empty_output><jupyter_text>Step 4. See the first 25 entries<jupyter_code>users.head(25)<jupyter_output><empty_output><jupyter_text>Step 5. See the last 10 entries<jupyter_code>users.tail(10)<jupyter_output><empty_output><jupyter_text>Step 6. What is the number of observations in the dataset?<jupyter_code>users.info() #read from the entries value
users.shape[0]<jupyter_output><empty_output><jupyter_text>Step 7. What is the number of columns in the dataset?<jupyter_code>users.shape[1]<jupyter_output><empty_output><jupyter_text>Step 8. Print the name of all the columns.<jupyter_code>users.columns<jupyter_output><empty_output><jupyter_text>Step 9. How is the dataset indexed?<jupyter_code>users.index<jupyter_output><empty_output><jupyter_text>Step 10. What is the data type of each column?<jupyter_code>users.dtypes<jupyter_output><empty_output><jupyter_text>Step 11. Print only the occupation column<jupyter_code>users.occupation<jupyter_output><empty_output><jupyter_text>Step 12. How many different occupations there are in this dataset?<jupyter_code>users.occupation.nunique()<jupyter_output><empty_output><jupyter_text>Step 13. What is the most frequent occupation?<jupyter_code>users['occupation'].value_counts().head(1).index[0]<jupyter_output><empty_output><jupyter_text>Step 14. Summarize the DataFrame.<jupyter_code>users.describe()<jupyter_output><empty_output><jupyter_text>Step 15. Summarize all the columns<jupyter_code>users.describe(include="all")<jupyter_output><empty_output><jupyter_text>Step 16. Summarize only the occupation column<jupyter_code>users.occupation.describe()<jupyter_output><empty_output><jupyter_text>Step 17. What is the mean age of users?<jupyter_code>round(users.age.mean(), ndigits=2)<jupyter_output><empty_output><jupyter_text>Step 18. What is the age with least occurrence?<jupyter_code>users.age.value_counts().tail(10)<jupyter_output><empty_output><jupyter_text>Ages 11, 10, 73, 66, and 7 all have only one observation.Below are examples for searching documentation on specific functions to better understand function arguments and behavior.<jupyter_code>#for a function within a library
pd.value_counts?
#for a function within base Python
len?
users?
#accessing source code of a function
pd.value_counts??<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10445370073694328, 'fraction_numerical': 0.01986542774751682, 'mean_word_length': 7.92, 'pattern_counts': {'":': 0, '<': 72, '<?xml version=': 0, '>': 72, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '508651', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1111, 'n_tokens_neox': 1048, 'n_words': 239, 'path': '01_Getting_&_Knowing_Your_Data/Occupation/Exercises_JW_01_16_20.ipynb', 'repo_name': 'williazo/pandas_exercises', 'size': '3121'}