<jupyter_start><jupyter_text>Ocean_Proximity is a categorical column. The rest of the Columns are numerical columns.  Null values are different from empty cells. Null values are shown with NaN. Look for shape of dataframe, data types of columns, and whether or not there are null values.<jupyter_code>#basic descriptive statistics for all numerical columns
df.describe(include='all')
#use df.describe(include='all') to see catagorical 
# pandas uses bin 10 when creating a histogram.<jupyter_output><empty_output><jupyter_text>When plotting a histogram, we are looking to find the distribution of data, center and spread, skewness, and presence of outliers.<jupyter_code>df.hist(figsize=(15,15))
df['median_income'].hist(bins=50)
df['ocean_proximity'].hist()
df['ocean_proximity'].value_counts()<jupyter_output><empty_output><jupyter_text>Boxplot -> the box is the middle 50%. The outliers are the dots that are on the oustide of the two lines.<jupyter_code>#creates an array of 10 numbers
x = np.linspace(0, 10, 10)
x
#adds two numbers, 20 and 22
x = np.append(x, [14, 22, 100])
x
#changes the array into a DataFrame
x = pd.DataFrame(x)
x.head()
x.boxplot()
df['median_income'].plot.box()
df['housing_median_age'].plot.box()
fig, ax = plt.subplots(2, figsize=(10, 20))
ax[0].scatter(x=df['median_income'], y=df['median_house_value'], alpha = 0.2) # alpha=0.2 specifies the opacity
ax[0].set_xlabel('median income')
ax[0].set_ylabel('median house value')
ax[1].scatter(x=df['housing_median_age'], y=df['median_house_value'], alpha=0.2) # alpha=0.2 specifies the opacity
ax[1].set_xlabel('housing median age')
ax[1].set_ylabel('median house value')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1282051282051282, 'fraction_numerical': 0.02802623732856291, 'mean_word_length': 6.697247706422019, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '609987', 'license': 'MIT', 'n_tokens_mistral': 622, 'n_tokens_neox': 563, 'n_words': 171, 'path': 'EDA-basics.ipynb', 'repo_name': 'Sroberhol/misy467-assignments', 'size': '1677'}