<jupyter_start><jupyter_text>DAT210x - Programming with Python for DS  Module2 - Lab4 Import and alias Pandas:<jupyter_code>import pandas as pd<jupyter_output><empty_output><jupyter_text>Load up the table from the link, and extract the dataset out of it. If you're having issues with this, look carefully at the sample code provided in the reading:<jupyter_code>df = pd.read_html('http://www.espn.com/nhl/statistics/player/_/stat/points/sort/points/year/2015/seasontype/2', header = 1)
df<jupyter_output><empty_output><jupyter_text>Next up, rename the columns so that they are _similar_ to the column definitions provided to you on the website. Be careful and don't accidentally use any column names twice. If a column uses special characters, you can replace them with regular characters to make it easier to work with:<jupyter_code>df = df[0].rename(columns={"+/-": "PM", "PTS/G": "PG", "G.1": "PPG", "A.1": "PPA" , "G.2": "SHG", "A.2": "SHA"})
# read_html returns a list of 1 element where the element is the dataframe 
df<jupyter_output><empty_output><jupyter_text>Get rid of any row that has at least 4 NANs in it. That is, any rows that do not contain player points statistics:<jupyter_code>df = df.dropna(axis=0, thresh=4)
df<jupyter_output><empty_output><jupyter_text>At this point, look through your dataset by printing it. There probably still are some erroneous rows in there. What indexing command(s) will you use to select all rows EXCEPT those rows?<jupyter_code>df = df.drop_duplicates(subset='PLAYER', keep=False).reset_index(drop=True)
df<jupyter_output><empty_output><jupyter_text>Get rid of the 'RK' column:<jupyter_code>df = df.drop(labels = 'RK', axis = 1)
df<jupyter_output><empty_output><jupyter_text>Make sure there are no holes in your index by resetting it. There is an example of this in the reading material. By the way, drop the original index.<jupyter_code># .. your code here ..<jupyter_output><empty_output><jupyter_text>Check the data type of all columns, and ensure those that should be numeric are numeric.<jupyter_code>df.GP = pd.to_numeric(df.GP,errors = 'coerce')
df.G = pd.to_numeric(df.G,errors = 'coerce')
df.A = pd.to_numeric(df.A,errors = 'coerce')
df.PTS = pd.to_numeric(df.PTS,errors = 'coerce')
df.PM = pd.to_numeric(df.PM,errors = 'coerce')
df.PIM = pd.to_numeric(df.PIM,errors = 'coerce')
df.PG = pd.to_numeric(df.PG,errors = 'coerce')
df.SOG = pd.to_numeric(df.SOG,errors = 'coerce')
df.PCT = pd.to_numeric(df.PCT,errors = 'coerce')
df.GWG = pd.to_numeric(df.GWG,errors = 'coerce')
df.PPG = pd.to_numeric(df.PPG,errors = 'coerce')
df.SHG = pd.to_numeric(df.SHG,errors = 'coerce')
df.PPA = pd.to_numeric(df.PPA,errors = 'coerce')
df.SHA = pd.to_numeric(df.SHA,errors = 'coerce')
df.dtypes<jupyter_output><empty_output><jupyter_text>Your dataframe is now ready! Use the appropriate commands to answer the questions on the course lab page.<jupyter_code>df.shape
len(df.PCT.unique())
df.loc[15, 'GP']+df.loc[16, 'GP']<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12733957219251338, 'fraction_numerical': 0.008355614973262033, 'mean_word_length': 5.802272727272728, 'pattern_counts': {'":': 6, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '541774', 'license': 'MIT', 'n_tokens_mistral': 1110, 'n_tokens_neox': 1066, 'n_words': 303, 'path': 'Module2/Module2 - Lab4.ipynb', 'repo_name': 'huangale/edxpy', 'size': '2992'}