<jupyter_start><jupyter_text>查看有多少空值<jupyter_code># HANDLING MISSING VALUES
print('There are %s NAN values in the train data'%train_file.isnull().sum().sum())<jupyter_output><empty_output><jupyter_text>![image.png](attachment:02251ce0-7f72-47ab-9af9-fdbe0bb72b22.png)  缺失值处理  创建一个filter整体填充缺失值<jupyter_code>features = [f'feature_{i}' for i in range(130)]
val_range = train_file[features].max()-train_file[features].min()
filler = pd.Series(train_file[features].min()-0.01*val_range, index=features)  #一个用负异常值填充所有缺失值的函数
def fill_missing(df):
    df[features] = df[features].fillna(filler)
    return df  

train = fill_missing(train_file)

train.info()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14537444933920704, 'fraction_numerical': 0.03524229074889868, 'mean_word_length': 8.342465753424657, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '49252', 'license': 'MIT', 'n_tokens_mistral': 301, 'n_tokens_neox': 306, 'n_words': 35, 'path': '机器学习/Kaggle代码/数据预处理.ipynb', 'repo_name': 'Yanie1asdfg/Quant-Lectures', 'size': '681'}