<jupyter_start><jupyter_text>Minimizing KL Divergence Let’s see how we could go about minimizing the KL divergence between two probability distributions using gradient descent. To begin, we create a probability distribution with a known mean (0) and variance (2). Then, we create another distribution with random parameters.<jupyter_code>import os
import warnings
warnings.filterwarnings('ignore')
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = (4,4) # Make the figures a bit bigger
plt.style.use('fivethirtyeight') 
import numpy as np
from scipy.stats import norm
import tensorflow as tf
import seaborn as sns
sns.set()

import math
from tqdm import tqdm
np.random.seed(7)<jupyter_output><empty_output><jupyter_text>To begin, we create a probability distribution, $p$, with a known mean (0) and variance (2).<jupyter_code>x = np.arange(-10, 10, 0.1)
x.shape[0]
tf_pdf_shape=(1, x.shape[0])
p = tf.placeholder(tf.float64, shape=tf_pdf_shape)#p_pdf.shape
#mu = tf.Variable(np.zeros(1))
#mu = tf.Variable(tf.truncated_normal((1,), stddev=3.0))
mu = tf.Variable(np.ones(1)*5)
print(mu.dtype)
varq = tf.Variable(np.eye(1))
print(varq.dtype)
normal = tf.exp(-tf.square(x - mu) / (2 * varq))
q = normal / tf.reduce_sum(normal)
learning_rate = 0.01
nb_epochs = 500*2<jupyter_output><empty_output><jupyter_text>We define a function to compute the KL divergence that excludes probabilities equal to zero.<jupyter_code>kl_divergence = tf.reduce_sum( p * tf.log(p / q))
kl_divergence = tf.reduce_sum(
    tf.where(p == 0, tf.zeros(tf_pdf_shape, tf.float64), p * tf.log(p / q))
)
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(kl_divergence)
init = tf.global_variables_initializer()
sess = tf.compat.v1.InteractiveSession()
sess.run(init)

history = []
means = []
variances = []<jupyter_output><empty_output><jupyter_text>Just for test<jupyter_code>m1 = 0
var1 = 2
p_pdf0 = norm.pdf(x, m1, np.sqrt(var1))
p_pdf1 = 1.0 / np.sqrt(var1) / np.sqrt(2 * math.pi) * np.exp(-np.square(x - m1) / (2 * var1))
import matplotlib
plt.plot(p_pdf0)
plt.plot(p_pdf1, marker=",")<jupyter_output><empty_output><jupyter_text>KL(P||Q) !!* $p$ : given (target)* $q$ : variables to learn Generating values for $p$<jupyter_code>m_truth = 0
var_truth = 7
p_pdf0 = norm.pdf(x, m_truth, np.sqrt(var_truth))
p_pdf0 = 1.0 / np.sqrt(var_truth) / np.sqrt(2 * math.pi) * np.exp(-np.square(x - m_truth) / (2 * var_truth))
p_pdf = p_pdf0.reshape(1, -1)
for i in tqdm(range(nb_epochs)):
    sess.run(optimizer, { p: p_pdf })
    history.append(sess.run(kl_divergence, { p: p_pdf }))
    means.append(sess.run(mu)[0])
    variances.append(sess.run(varq)[0][0])

    if i % 100 == 10:
        print(sess.run(mu)[0], sess.run(varq)[0][0])<jupyter_output>11%|█         | 107/1000 [00:00<00:01, 514.85it/s]<jupyter_text>Plot the results<jupyter_code>len1 = np.shape(means)[0]
alphas = np.linspace(0.1, 1, len1)
rgba_colors = np.zeros((len1,4))
# for red the first column needs to be one
rgba_colors[:,0] = 1.0
# the fourth column needs to be your alphas
rgba_colors[:, 3] = alphas
print(rgba_colors.shape)
grange = range(len1)
print(np.shape(grange))
for mean, variance, g in zip(means, variances, grange):
    if g%5 ==0:
        q_pdf = norm.pdf(x, mean, np.sqrt(variance))
        plt.plot(x, q_pdf.reshape(-1, 1), color=rgba_colors[g])
    
plt.title('KL(P||Q) = %1.3f' % history[-1])
plt.plot(x, p_pdf.reshape(-1, 1), linewidth=3)
plt.show()
#target
plt.plot(x, p_pdf.reshape(-1, 1), linewidth=5)

#initial
q_pdf = norm.pdf(x, means[0] , np.sqrt(variances[0]))
plt.plot(x, q_pdf.reshape(-1, 1))

#final
q_pdf = norm.pdf(x, means[-1] , np.sqrt(variances[-1]))
plt.plot(x, q_pdf.reshape(-1, 1), color='r')
plt.plot(means)
plt.xlabel('epoch')
plt.ylabel('mean')
plt.plot(variances)
plt.xlabel('epoch')
plt.ylabel('variances')
plt.plot(history)
plt.title('history')
plt.show()
#sess.close()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1644080797749936, 'fraction_numerical': 0.033750958834057786, 'mean_word_length': 5.170347003154574, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '26354', 'license': 'MIT', 'n_tokens_mistral': 1669, 'n_tokens_neox': 1584, 'n_words': 338, 'path': 'Optimization/KLDivergence/KL_divergence_optimization.ipynb', 'repo_name': 'dalek7/Algorithms', 'size': '3911'}