<jupyter_start><jupyter_text>Original paper: https://arxiv.org/abs/1806.00035  0. Read real and generated images<jupyter_code>#!L
import math
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.lines as mlines
import matplotlib
#!L
CHANNEL_NUM = 3
PICTURE_SIZE = 64
#!L
class ParticleDataset():
    def __init__(self, file):
        self.data = np.load(file)
        self.image = self.data['Pictures'].reshape(-1, CHANNEL_NUM*PICTURE_SIZE*PICTURE_SIZE)

    def __len__(self):
        return len(self.image)

    def __getitem__(self, i):
        return {
            "Pictures": self.image[i],
        }
#!L
real_data = ParticleDataset('real.npz')
vae_data = ParticleDataset('vae.npz')
gan_data = ParticleDataset('gan.npz')<jupyter_output><empty_output><jupyter_text>Make sure that the values of real and generated data are of the same order - it is important for cooperative binarizing<jupyter_code>#!L
print(np.min(real_data.image), np.max(real_data.image))
print(np.min(gan_data.image), np.max(gan_data.image))
print(np.min(vae_data.image), np.max(vae_data.image))<jupyter_output><empty_output><jupyter_text>1. Binarize  To understand how real and generated objects are close to each other, we need to choose a space of features in which we look these objects at We go the easiest way and take pixels' values as features.<jupyter_code>#!L
from sklearn.cluster import KMeans, MiniBatchKMeans
import math
## function which map object to probability distribution ##

def bin_counts (real_data, generated_data, number_of_bins=25):
    # binirize real and generated data, plot histogram and found density function
    return real_density, gen_density<jupyter_output><empty_output><jupyter_text>Create $\alpha-$ and $\beta-$ vectors as in$\hat{PRD}(Q,P) = \{(\alpha(\lambda), \beta(\lambda))| \lambda \in \Lambda \}$, where $\Lambda = \{\tan (\frac{i}{m+1} \frac{\pi}{2}) | i = 1, 2 ... m\}$<jupyter_code>#!L
def count_alpha_beta (real_density, gen_density, num_angles = 1000):
    assert real_density.shape == gen_density.shape
    alpha_vec = []
    beta_vec = []
    angles = np.linspace(1e-6, np.pi/2 - 1e-6, num=num_angles)
    # you code
    return alpha_vec, beta_vec<jupyter_output><empty_output><jupyter_text>For stability, take the average of several repetitions<jupyter_code>#!L
def count_prd(reals, gens, repeat_number = 10):
    vectors = [count_alpha_beta(reals, gens) for i in range(repeat_number)]
    vectors = np.array(vectors).mean(axis=0)
    print (vectors.shape)
    return vectors<jupyter_output><empty_output><jupyter_text>2. Apply it<jupyter_code>#!L
a, b = bin_counts(real_data.image, vae_data.image)
c, d = bin_counts(real_data.image, gan_data.image)<jupyter_output><empty_output><jupyter_text>3. Make vectors for plot and plot<jupyter_code>#!L
data_for_plots = count_prd(a, b)
data_for_plots2 = count_prd(c, d)
#!L
fig = plt.figure(figsize=(2.5, 2.5), dpi=200)
fig.add_subplot(111).tick_params(axis='both', which='major', labelsize=8)
plt.xlim([0, 1])
plt.ylim([0, 1])
plt.xlabel('Recall', fontsize=12)
plt.ylabel('Precision', fontsize=12)
plt.plot(data_for_plots[0], data_for_plots[1], label = "VAE")
plt.plot(data_for_plots2[0], data_for_plots2[1], label = "GAN")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>**What curves were obtained for the first(VAE) and the second(GAN) models? What can we say about the advantages and disadvantages of each model?**<jupyter_code>#type answer here<jupyter_output><empty_output><jupyter_text>Bonus: about features' spaceIt is possible to transfer the picture-> embedding, for example, using the 1st part of the Inception network as a feature extraxtor. This embedding can be used for bin counts also<jupyter_code>#!L
# if you came here and still alive, the implementation of idea above will give you extra points =)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12415584415584416, 'fraction_numerical': 0.016103896103896103, 'mean_word_length': 5.494097807757167, 'pattern_counts': {'":': 1, '<': 37, '<?xml version=': 0, '>': 38, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '301823', 'license': 'MIT', 'n_tokens_mistral': 1410, 'n_tokens_neox': 1346, 'n_words': 345, 'path': 'homework03/homework03_part3b_prd_score_advanced.ipynb', 'repo_name': 'VendettaPrime/Practical_DL', 'size': '3850'}