<jupyter_start><jupyter_text>Dict对象来实现文本的统计处理<jupyter_code># init a sentence including many words
sentence = "The ability to analyze data with Python is critical in data science Learn the basics and move on to create stunning visualizations"
# init a dict
word_dict = {}

# analysis the words of the sentence and statistic the number of every word
for word in sentence.split():
    if word not in word_dict:
        word_dict[word] = 1
    else:
        word_dict[word] += 1
        
# print the result
print word_dict<jupyter_output>{'and': 1, 'on': 1, 'basics': 1, 'ability': 1, 'move': 1, 'Python': 1, 'data': 2, 'is': 1, 'Learn': 1, 'stunning': 1, 'to': 2, 'create': 1, 'critical': 1, 'in': 1, 'visualizations': 1, 'The': 1, 'analyze': 1, 'with': 1, 'the': 1, 'science': 1}<jupyter_text>使用另外的一个defaultdict类，需要引入collections包<jupyter_code>from collections import defaultdict

sentence = "The ability to analyze data with Python is critical in data science Learn the basics and move on to create stunning visualizations"

word_dict = defaultdict(int)

for word in sentence.split():
    word_dict[word] +=1
    
print word_dict<jupyter_output>defaultdict(<type 'int'>, {'and': 1, 'on': 1, 'basics': 1, 'ability': 1, 'move': 1, 'Python': 1, 'data': 2, 'is': 1, 'Learn': 1, 'stunning': 1, 'to': 2, 'create': 1, 'critical': 1, 'in': 1, 'visualizations': 1, 'The': 1, 'analyze': 1, 'with': 1, 'the': 1, 'science': 1})<jupyter_text>变量使用dict的items()函数，keys()可以遍历所有的键，values()可以遍历所有的值<jupyter_code>for key,value in word_dict.items():
    print key,value<jupyter_output><empty_output><jupyter_text>counter是一个字典子类，用来统计键值对类型的对象 from collections import Countersentence = "The ability to analyze data with Python is critical in data science Learn the basics and move on to create stunning visualizations"words = sentence.split()word_dict = Counter(words)print word_dict 使用字典的字典，举一个例子，有三个用户的五个电影的评分，则用dict如何存储，就是字典的嵌套<jupyter_code>from collections import defaultdict

user_movie_rating = defaultdict(lambda:defaultdict(int))

# 初始化一个用户的电影评分

user_movie_rating['Roc-J']['wolf1'] = 4
user_movie_rating['Roc-J']['wolf2'] = 5
user_movie_rating['Roc-J']['wolf3'] = 3
user_movie_rating['Roc-J']['icecream'] = 3
user_movie_rating['Roc-J']['SW'] = 5

print user_movie_rating<jupyter_output>defaultdict(<function <lambda> at 0x04B68D70>, {'Roc-J': defaultdict(<type 'int'>, {'icecream': 3, 'SW': 5, 'wolf1': 4, 'wolf3': 3, 'wolf2': 5})})
{'fraction_non_alphanumeric': 0.14989648033126293, 'fraction_numerical': 0.02732919254658385, 'mean_word_length': 6.2119402985074625, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '469542', 'license': 'Apache-2.0', 'n_tokens_mistral': 985, 'n_tokens_neox': 919, 'n_words': 251, 'path': '001_dict_for_statistic_text.ipynb', 'repo_name': 'Roc-J/Python_data_science', 'size': '2415'}