<jupyter_start><jupyter_text>Original data path is on ../vqvae/data.npy and ../vqvae/labels.npy                 (2080, 128, 128, 3)Generated data from 5 operations is on ../cancer_deep_learning/new_data.npy and    (10400, 128, 128, 3)                                       ../cancer_deep_learning/new_labels.npy<jupyter_code>original_data     = np.load('../vqvae/data.npy')
generated_5o_data = np.load('../cancer_deep_learning/new_data.npy')
IPython.display.display(PIL.Image.fromarray(original_data[0]))
#print(labels[0])
if not os.path.exists('original'):
    os.mkdir('original')
for i in range(len(original_data)):
    path = 'original/' + str(i) + '.png'
    imageio.imwrite(path, original_data[i])
if not os.path.exists('generated_5o_data'):
    os.mkdir('generated_5o_data')
for i in range(len(generated_5o_data)):
    path = 'generated_5o_data/' + str(i) + '.png'
    imageio.imwrite(path, generated_5o_data[i])
path       = ['original','generated_5o_data']
batch_size = 20
gpu        = True
dims       = 2048
fid_value = calculate_fid_given_paths(path, batch_size, gpu, dims)
print('FID: ', fid_value)<jupyter_output>Downloading: "https://github.com/mseitzer/pytorch-fid/releases/download/fid_weights/pt_inception-2015-12-05-6726825d.pth" to /home/tone/.cache/torch/checkpoints/pt_inception-2015-12-05-6726825d.pth
100%|██████████| 95628359/95628359 [02:33<00:00, 621265.14it/s]
100%|██████████| 104/104 [00:06<00:00, 16.51it/s]
100%|██████████| 520/520 [00:31<00:00, 16.68it/s]
{'fraction_non_alphanumeric': 0.1768661735036987, 'fraction_numerical': 0.09818426361802286, 'mean_word_length': 4.13103448275862, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 4, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '262109', 'license': 'Apache-2.0', 'n_tokens_mistral': 702, 'n_tokens_neox': 582, 'n_words': 88, 'path': 'fid_from_5o_dataset.ipynb', 'repo_name': 'superizer/pytorch-fid', 'size': '1487'}