<jupyter_start><jupyter_text>图像预处理，图像路径获取 读取类别，子路径获取<jupyter_code>from jpg_data_preprocess import *

# 来源
train_dir = 'pig_pic_src_processed_1_part2/train/'
valid_dir = 'pig_pic_src_processed_1_part2/valid/'

# 目的
processed_train_dir = 'pig_pic_src_processed_combine/train/'
processed_valid_dir = 'pig_pic_src_processed_combine/valid/'<jupyter_output><empty_output><jupyter_text>图像预处理    jpg -> process -> jpg<jupyter_code>preprocess_all_pics(train_dir, processed_train_dir, random_crop_window = [448, 448], data_plus = 10, train = True)
preprocess_all_pics(valid_dir, processed_valid_dir, train = False)<jupyter_output>pig_pic_src_processed_1_part1_1/train/03.pig -> pig_pic_src_processed_combine/train/03.pig#  start process
10 images processed
20 images processed
30 images processed
40 images processed
50 images processed
60 images processed
70 images processed
80 images processed
90 images processed
100 images processed
110 images processed
120 images processed
130 images processed
pig_pic_src_processed_1_part1_1/train/04.pig -> pig_pic_src_processed_combine/train/04.pig#  start process
10 images processed
20 images processed
30 images processed
40 images processed
50 images processed
60 images processed
70 images processed
80 images processed
90 images processed
100 images processed
110 images processed
120 images processed
130 images processed
140 images processed
142 images processed
pig_pic_src_processed_1_part1_1/valid/03.pig -> pig_pic_src_processed_combine/valid/03.pig#  start process
4 images processed
pig_pic_src_processed_1_part1_1/valid/04.pig -> pig_pic_src_processed_combine/valid/04.pig[...]
{'fraction_non_alphanumeric': 0.06276923076923077, 'fraction_numerical': 0.06584615384615385, 'mean_word_length': 7.338461538461538, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '576305', 'license': 'MIT', 'n_tokens_mistral': 627, 'n_tokens_neox': 569, 'n_words': 129, 'path': 'tf-vgg16-new/Transfer_Learning_data_preprocess_2.ipynb', 'repo_name': 'LinMaris/JDD_PigImageClassification', 'size': '1625'}