<jupyter_start><jupyter_text>**We can load and read data as a csv file. Data includes median housholds income in San Mateo county.**<jupyter_code>df = pd.read_json("https://data.smcgov.org/resource/v6r6-44a3.json")
df.head(10)<jupyter_output><empty_output><jupyter_text>**We can also open data as csv file.**<jupyter_code>df2 = pd.read_csv("https://data.smcgov.org/resource/v6r6-44a3.csv", index_col=0)
df2.head(10)
'''Get the statistics'''

df.describe()
"""Drop a column"""

df.drop('one_year_change', axis=1)<jupyter_output><empty_output><jupyter_text>**If we drop a column, it doesn't change the original data frame by default.**<jupyter_code>df<jupyter_output><empty_output><jupyter_text>**If we want to change the original dataframe we should use inplace=True.**<jupyter_code>df.drop('one_year_change', axis=1, inplace=True)
df
df.index[[1, 5, 10]]<jupyter_output><empty_output><jupyter_text>**We can delete multiple rows**<jupyter_code>df.drop([0, 1], axis=0)<jupyter_output><empty_output><jupyter_text>**We can also delete multiple rows by index.**<jupyter_code>df.drop(df.index[[0,1,2]])<jupyter_output><empty_output><jupyter_text>**We can add a new column**<jupyter_code>df['house square feet'] = None
df<jupyter_output><empty_output><jupyter_text>**We can fill 'None' values by fillna**<jupyter_code>df.fillna(1000, inplace=True)
df<jupyter_output><empty_output><jupyter_text>**We can replace values in a dataframe**<jupyter_code>df.loc[[0,1,2,3],'house square feet'] = [1100, 1500, 2000, 3000]
df
df.isnull().sum()<jupyter_output><empty_output><jupyter_text>**We can also drop duplicate rows**<jupyter_code>df.drop_duplicates()<jupyter_output><empty_output><jupyter_text>**And we can drop any columns with missing values:**<jupyter_code>df['house square feet'] = None
df
df.dropna(inplace=True, axis=1)
df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15538711423930698, 'fraction_numerical': 0.029236599891716297, 'mean_word_length': 9.266666666666667, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '464158', 'license': 'MIT', 'n_tokens_mistral': 746, 'n_tokens_neox': 712, 'n_words': 134, 'path': 'Pandas_Python/pandas - data cleaning.ipynb', 'repo_name': 'mdaneshv/DataScience', 'size': '1847'}