<jupyter_start><jupyter_text>4 - Correct TypesIn this fourth step I'll show you how to fix incorrect types in the category column.You can use this process to clean unexpected character types in other columns as well.<jupyter_code>import pandas as pd 

df = pd.read_json("customer_data.json", convert_dates=False)
df.head()
# taking a look at unique values
print(df["category"].unique())<jupyter_output>['household' 'clothing' 'outdoor' 'electronics' 'appliances' 'house'
 'elect^ronics' '^electro$nics' 'outdo&or' 'household_' '?out$door' 'elec'
 'app' 'house_hold' '%appliances' '\\appliances' 'electronic']<jupyter_text>Taking a look at the unique values we can see some unexpected characters such as % and \\.We can clean these by putting these unexpected characters in a list called bad_characters.We'll loop through the rows in the category column and if we find an unexpected character we'll replace it with empty space.<jupyter_code>bad_characters = ["^", "%", "&", "?", "$", '\\''']

cnt = 0
for row in df["category"]:
    for character in bad_characters:
        row = row.replace(character,"")
    df.loc[cnt, "category"] = row
    cnt+=1

print(df["category"].unique())<jupyter_output>['household' 'clothing' 'outdoor' 'electronics' 'appliances' 'house'
 'household_' 'elec' 'app' 'house_hold' 'electronic']
{'fraction_non_alphanumeric': 0.13505311077389984, 'fraction_numerical': 0.002276176024279211, 'mean_word_length': 5.628140703517588, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '367299', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 427, 'n_tokens_neox': 412, 'n_words': 139, 'path': '4 - Correct Types.ipynb', 'repo_name': 'NickScherer/example-NickScherer.github.io', 'size': '1318'}