<jupyter_start><jupyter_text>数据属性简介。 |  属性  |                  描述    ||:------:|:--------------------------------------:|| `ID` | 自增字段 || `Movie_Name` | 电影名 || `Movie_Score` | 电影总体评分 || `Review_Count`  | 评论量统计 || `Movie_Star_Distribution` | 电影的评分分布 || `Collect_Date` |  数据收集时间 || `Username` | 用户昵称 || `Post_Date` | 评论发表日期 || `Score` | 用户评分 || `User_Comment` | 用户评论 || `User_Comment_Distribution` | 用户评分情况 || `Comment_Like_Count` | 用户评论点赞量 |  数据 EDA  按照电影名个用户名字进行去重，并保留首次出现的数据。<jupyter_code>ratings_explicit.drop_duplicates(
    subset=['Movie_Name', 'Username'], keep='first', inplace=True)<jupyter_output><empty_output><jupyter_text>查看数据统计描述。<jupyter_code>ratings_explicit.describe()<jupyter_output><empty_output><jupyter_text>查看数据结构。<jupyter_code>ratings_explicit.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 10000 entries, 0 to 9999
Data columns (total 13 columns):
ID                           10000 non-null int64
Movie_Name                   10000 non-null object
Movie_Score                  10000 non-null float64
Review_Count                 10000 non-null int64
Movie_Star_Distribution      10000 non-null object
Collect_Date                 10000 non-null object
Username                     9999 non-null object
Post_Date                    10000 non-null object
Score                        10000 non-null int64
User_Comment                 10000 non-null object
User_Comment_Distribution    10000 non-null object
Comment_Like_Count           10000 non-null int64
Movie_Tags                   10000 non-null object
dtypes: float64(1), int64(4), object(8)
memory usage: 1.1+ MB<jupyter_text>查看总体用户数量。<jupyter_code>len(ratings_explicit.Username.unique())<jupyter_output><empty_output><jupyter_text>查看总体电影数量。<jupyter_code>len(ratings_explicit.Movie_Name.unique())<jupyter_output><empty_output><jupyter_text>统计观看电影最多的 topk 的用户。<jupyter_code>topk = 10
ratings_explicit['Username'].value_counts()[:topk]<jupyter_output><empty_output><jupyter_text>查看前 topk 个用户的观影分布。<jupyter_code>import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
topk = 100
number_count = ratings_explicit['Username'].value_counts()
plt.bar(range(len(number_count[:topk])), number_count[:topk])
plt.show()<jupyter_output><empty_output><jupyter_text>统计观看人数 topk 的电影。<jupyter_code>topk = 10
ratings_explicit['Movie_Name'].value_counts()[:topk]<jupyter_output><empty_output><jupyter_text>数据可视化 用户对电影的打分情况。<jupyter_code>sns.countplot(data=ratings_explicit, x='Score')
plt.show()<jupyter_output><empty_output><jupyter_text>提取用户评价时间和统计信息。<jupyter_code>post_time, post_count = [], []

for i in ratings_explicit['Post_Date'].groupby(
        ratings_explicit['Post_Date']).count().items():
    # 取出时间标签
    post_time.append(i[0])
    # 取出计数信息
    post_count.append(i[1])<jupyter_output><empty_output><jupyter_text>用户评论时间序列 使用折线图对用户评论时间进行可视化。<jupyter_code>!pip install pyecharts==1.6.2
from pyecharts.charts import Line
from pyecharts import options as opts

# 加载图的配置，长宽的参数设置
line = Line(init_opts=opts.InitOpts(width='660px', height='350px'))

# 设置横坐标为时间
line.add_xaxis(post_time)

# 设置纵坐标为用户评论条数
line.add_yaxis("", post_count)

# 设置标题
line.set_global_opts(title_opts=opts.TitleOpts(title='用户评论时间序列图'))

# 设置显示统计量
line.set_series_opts(label_opts=opts.LabelOpts(is_show=True), )
line.render_notebook()<jupyter_output><empty_output><jupyter_text>用户评论关键词云<jupyter_code>from jieba import analyse

# 默认提取前 20 个关键词
def get_key_words(comment, topK=20):
    return analyse.textrank(comment, topK, withWeight=False, allowPOS=('n'))
# 使用测试样例进行测试。
comment = '电影院环境不错'
get_key_words(comment)<jupyter_output>Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Loading model cost 0.795 seconds.
Prefix dict has been built successfully.<jupyter_text>提取影评中的 topK 关键词，数据量有些大，请耐心等待。<jupyter_code>from tqdm.notebook import tqdm

keys_words_list = []
for comment in tqdm(ratings_explicit['User_Comment']):
    keys_words_list += get_key_words(comment)
# 加载词频统计模块
from sklearn.feature_extraction.text import CountVectorizer

# 设定 topn 的高频词语阈值，可以根据需要进行改变大小
topn = 500

# 加载统计模块
count_extract = CountVectorizer(max_features=topn)

# 开始对语料进行统计
count_extract.fit_transform(keys_words_list)

# 输入数据中的每一个词为（word，value）这样的元祖形式，然后将所有的词放入一个list中
comment_count_pair = [(k, int(v))
                      for k, v in count_extract.vocabulary_.items()]
# 加载词云模块
from pyecharts.charts import WordCloud

comment_wordcloud = WordCloud()

# word_size_range 表示词语大小范围，可以根据需要改变
comment_wordcloud.add(series_name="",
                      data_pair=comment_count_pair,
                      word_size_range=[5, 100])

# 设置词云的全局配置
comment_wordcloud.set_global_opts(
    title_opts=opts.TitleOpts(
        title="用户评论关键词云分析",
        # 设置标题文字大小
        title_textstyle_opts=opts.TextStyleOpts(font_size=15)),
    # 显示词云中的词频数字
    tooltip_opts=opts.TooltipOpts(is_show=True),)

# 直接在 notebook 中显示词云
comment_wordcloud.render_notebook()<jupyter_output><empty_output><jupyter_text>根据用户评论对电影进行标签化。<jupyter_code>movie_list = ratings_explicit['Movie_Name'].unique()
movie_list[0]<jupyter_output><empty_output><jupyter_text>批量从用户评论中提取电影关键词。<jupyter_code>movie_map = dict()
for movie_name in tqdm(movie_list):
    comment_text = ratings_explicit[ratings_explicit['Movie_Name']==movie_name]['User_Comment'].values
    # 构建电影关键词标签
    movie_map[movie_name] = get_key_words(str(comment_text))<jupyter_output><empty_output><jupyter_text>为电影打上用户评价标签。<jupyter_code>ratings_explicit['Movie_Tags'] = ratings_explicit['Movie_Name'].apply(lambda x : movie_map[x])
ratings_explicit.head(1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1200783894530554, 'fraction_numerical': 0.023873151612328523, 'mean_word_length': 5.386803185437998, 'pattern_counts': {'":': 0, '<': 64, '<?xml version=': 0, '>': 64, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '225964', 'license': 'Apache-2.0', 'n_tokens_mistral': 2468, 'n_tokens_neox': 2516, 'n_words': 289, 'path': 'Data_EDA.ipynb', 'repo_name': 'CarryChang/RS_APP', 'size': '5613'}