<jupyter_start><jupyter_text>通过给定目录，统计所有的不同子文件类型及占用内存<jupyter_code>size_dict={}
type_dict={}
def get_size_type(path):
    files=os.listdir(path)
    for filename in files:
        temp_path=os.path.join(path,filename)
        if os.path.isdir(temp_path):
            get_size_type(temp_path)
        elif os.path.isfile(temp_path):
            # 获取文件后缀
            type_name=os.path.splitext(temp_path)
            # 无后缀名的文件
            if not type_name:
                type_dict.setdefault("None",0)
                type_dict['None']+=1
                size_dict.setdefault('None',0)
                size_dict['None']+=os.path.getsize(temp_path)
            # 有后缀的文件
            else:
                type_dict.setdefault(type_name,0)
                type_dict[type_name]+=1
                size_dict.setdefault(type_name,0)
                size_dict[type_name]+=os.path.getsize(temp_path)
path='./data'
get_size_type(path)
for each_type in type_dict.keys():
    print ("%5s下共有【%5s】的文件【%5d】个,占用内存【%7.2f】MB" %     
            (path,each_type,type_dict[each_type],\
            size_dict[each_type]/(1024*1024)))
print("总文件数:  【%d】"%(sum(type_dict.values())))
print("总内存大小:【%.2f】GB"%(sum(size_dict.values())/(1024**3)))<jupyter_output>./data下共有【('./data/data10954/cat_data/cat_12_train/JwmiPT6hsFfCbnjxQgAIyRz2t5Od3SkZ', '.jpg')】的文件【    1】个,占用内存【   0.09】MB
./data下共有【('./data/data10954/cat_data/cat_12_train/mEa7fyHbXCpOe21WLsAnYQzdU9ql8JVk', '.jpg')】的文件【    1】个,占用内存【   0.04】MB
./data下共有【('./data/data10954/cat_data/cat_12_train/zD5LwfW3i1Ed8M0NlgytnmCAUTaKehGB', '.jpg')】的文件【    1】个,占用内存【   0.01】MB
./data下共有【('./data/data10954/cat_data/cat_12_train/miO3kS0G9efcbg2IdTJH8KwzQsnEupYv', '.jpg')】的文件【    1】个,占用内存【   0.03】MB
./data下共有【('./data/data10954/cat_data/cat_12_train/PMGhgEmR26aYOLBiz1uIt50j8xKXNQeb', '.jpg')】的文件【    1】个,占用内存【   0.11】MB
./data下共有【('./data/data10954/cat_data/cat_12_train/0yTr3fswKBv4M8Fo2NcUnzibx6ClIm5e', '.jpg')】的文件【    1】个,占用内存【   0.09】MB
./data下共有【('./data/data10954/cat_data/cat_12_train/ZjyRN6We4KPG1YwStB9q5J7CpxzsAlFM', '.jpg')】的文件【    1】个,占用内存【   0.10】MB
./data下共有【('./data/data10954/cat_data/cat_12_train/2QMb8NpUA7wSLtYWD6fe1RFBhuVCxgd9', '.jpg')】的文件【    1】个,占用内存【   0.17】MB
./data下共有【('./data/data1[...]
{'fraction_non_alphanumeric': 0.1713520749665328, 'fraction_numerical': 0.07050423917893797, 'mean_word_length': 3.3703703703703702, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '178773', 'license': 'MIT', 'n_tokens_mistral': 1153, 'n_tokens_neox': 1158, 'n_words': 76, 'path': 'Practice1-10/1_MassiveFileTraverse.ipynb', 'repo_name': 'preccrep/ML-Homework-Python', 'size': '2241'}