<jupyter_start><jupyter_text>3 - Consistent StringsIn this third step I'll show you how to fix inconsistent strings in the category column.You can use: comment of action, python coding and results.<jupyter_code>import pandas as pd 

df = pd.read_json("customer_data.json", convert_dates=False)
df.head()<jupyter_output><empty_output><jupyter_text>Lets take a quick look at the data using the .unique() method.<jupyter_code>print(df["category"].unique())<jupyter_output>['household' 'clothing' 'outdoor' 'electronics' 'appliances' 'house'
 'elect^ronics' '^electro$nics' 'outdo&or' 'household_' '?out$door' 'elec'
 'app' 'house_hold' '%appliances' '\\appliances' 'electronic']<jupyter_text>Not all of the strings have consistent formats.The household category shows up with the following inconsistent formats ["household_", "house", "house_hold"].<jupyter_code>inconsistent_format = ["household_", "house", "house_hold"]

cnt = 0

for row in df["category"]:
    if row in inconsistent_format:
        df.loc[cnt, "category"] = "household"
    cnt+=1
    
print(df["category"].unique())<jupyter_output>['household' 'clothing' 'outdoor' 'electronics' 'appliances'
 'elect^ronics' '^electro$nics' 'outdo&or' '?out$door' 'elec' 'app'
 '%appliances' '\\appliances' 'electronic']
{'fraction_non_alphanumeric': 0.1470125786163522, 'fraction_numerical': 0.0023584905660377358, 'mean_word_length': 6.8580246913580245, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '369141', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 444, 'n_tokens_neox': 428, 'n_words': 111, 'path': '3 - Consistent Strings.ipynb', 'repo_name': 'NickScherer/example-NickScherer.github.io', 'size': '1272'}