<jupyter_start><jupyter_text>垃圾邮件分类https://www.kaggle.com/uciml/sms-spam-collection-dataset  读取数据<jupyter_code>import pandas as pd
import numpy as np 

data_dir = "../input/"

df = pd.read_csv(data_dir + '/spam.csv', encoding='latin-1')  
# 编码相关阅读http://blog.csdn.net/robertcpp/article/details/7837712 

# 查看数据
df.head()
# 查看v2的样本
# 查看v1的样本
# 查看数据的纬度<jupyter_output><empty_output><jupyter_text>把数据拆分成为训练集和测试集<jupyter_code>from sklearn.model_selection import train_test_split<jupyter_output><empty_output><jupyter_text>构建模型所需要的数据格式：一个词汇表，以及训练及测试数据的计数信息：(句子id,单词id)->计数<jupyter_code>from sklearn.feature_extraction.text import CountVectorizer
# 调用库来构造分类器所需的输入数据


# 用训练集的单词来建立词汇表<jupyter_output><empty_output><jupyter_text>图形化展示一些数据，获得更直观的理解<jupyter_code>import matplotlib.pyplot as plt # 画图常用库

#print(vectorizer.get_feature_names())
#统计每个单词出现的次数

# 排序：单词出现次数从高到低<jupyter_output><empty_output><jupyter_text>进行模型训练以及预测<jupyter_code>from sklearn.naive_bayes import MultinomialNB<jupyter_output><empty_output><jupyter_text>计算模型的准确率<jupyter_code>from sklearn.metrics import accuracy_score<jupyter_output><empty_output><jupyter_text>其他常用指标: （Naive Bayes 第二节课会补充）<jupyter_code>from sklearn.metrics import classification_report,confusion_matrix<jupyter_output><empty_output><jupyter_text>交叉验证的示范:<jupyter_code>from sklearn.model_selection import cross_val_score<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10701373825018076, 'fraction_numerical': 0.0072306579898770785, 'mean_word_length': 11.468468468468469, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 34, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '645337', 'license': 'MIT', 'n_tokens_mistral': 685, 'n_tokens_neox': 730, 'n_words': 51, 'path': 'naive_bayes-master/email/code/nb-sklearn-framework.ipynb', 'repo_name': '052D/Machine_Learning_Projects', 'size': '1383'}