<jupyter_start><jupyter_text>Some of the important preprocessing that are most useful are as follows:- Binarization- Mean Removal- Scaling- Normalization<jupyter_code>#Binarization
preprocessing.binarize(input_data,threshold=3)
#Mean Removal
print(preprocessing.scale(input_data))
#_scaler = preprocessing.MinMaxScaler()
print(preprocessing.minmax_scale(input_data,feature_range=(1,3)))
scalar_ = preprocessing.MinMaxScaler(feature_range=(1,3))
scalar_.fit_transform(input_data)
data_normalized_l1 = preprocessing.normalize(input_data, norm='l1') 
print(data_normalized_l1)
data_normalized_l2 = preprocessing.normalize(input_data, norm='l2') 
print(data_normalized_l2)
input_labels = ['red', 'black', 'red', 'green', 'black', 'yellow', 'white'] 
encoder = preprocessing.LabelEncoder().fit(input_labels)
print(encoder.classes_)
print(encoder.transform(['red']))<jupyter_output>['black' 'green' 'red' 'white' 'yellow']
[2]
{'fraction_non_alphanumeric': 0.13804347826086957, 'fraction_numerical': 0.013043478260869565, 'mean_word_length': 10.96103896103896, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '138217', 'license': 'MIT', 'n_tokens_mistral': 316, 'n_tokens_neox': 312, 'n_words': 52, 'path': '01_Preprocessing_Workout.ipynb', 'repo_name': 'fakabbir/AI-Playbook', 'size': '920'}