<jupyter_start><jupyter_text>Batch TrainingRunning algorithms which require the full data set for each updatecan be expensive when the data is large. In order to scale inferences,we can do _batch training_. This trains the model usingonly a subsample of data at a time.In this tutorial, we extend the[supervised learning tutorial](http://edwardlib.org/tutorials/supervised-regression), where the task is to infer hidden structure fromlabeled examples $\{(x_n, y_n)\}$.A webpage version is available athttp://edwardlib.org/tutorials/batch-training.<jupyter_code>from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import edward as ed
import numpy as np
import tensorflow as tf

from edward.models import Normal<jupyter_output><empty_output><jupyter_text>DataSimulate $N$ training examples and a fixed number of test examples.Each example is a pair of inputs $\mathbf{x}_n\in\mathbb{R}^{10}$ andoutputs $y_n\in\mathbb{R}$. They have a linear dependence withnormally distributed noise.We also define a helper function to select the next batch of datapoints from the full set of examples. It keeps track of the currentbatch index and returns the next batch using the function ``next()``. We will generate batches from `data` during inference.<jupyter_code>def build_toy_dataset(N, w):
  D = len(w)
  x = np.random.normal(0.0, 2.0, size=(N, D))
  y = np.dot(x, w) + np.random.normal(0.0, 0.05, size=N)
  return x, y
def generator(arrays, batch_size):
  """Generate batches, one with respect to each array's first axis."""
  starts = [0] * len(arrays)  # pointers to where we are in iteration
  while True:
    batches = []
    for i, array in enumerate(arrays):
      start = starts[i]
      stop = start + batch_size
      diff = stop - array.shape[0]
      if diff <= 0:
        batch = array[start:stop]
        starts[i] += batch_size
      else:
        batch = np.concatenate((array[start:], array[:diff]))
        starts[i] = diff
      batches.append(batch)
    yield batches
ed.set_seed(42)

N = 10000  # size of training data
M = 128    # batch size during training
D = 10     # number of features

w_true = np.ones(D) * 5
X_train, y_train = build_toy_dataset(N, w_true)
X_test, y_test = build_toy_dataset(235, w_true)

data = generator([X_train, y_train], M)<jupyter_output><empty_output><jupyter_text>ModelPosit the model as Bayesian linear regression (Murphy, 2012).For a set of $N$ data points $(\mathbf{X},\mathbf{y})=\{(\mathbf{x}_n, y_n)\}$,the model posits the following distributions:\begin{align*}  p(\mathbf{w})  &=  \text{Normal}(\mathbf{w} \mid \mathbf{0}, \sigma_w^2\mathbf{I}),  \\[1.5ex]  p(b)  &=  \text{Normal}(b \mid 0, \sigma_b^2),  \\  p(\mathbf{y} \mid \mathbf{w}, b, \mathbf{X})  &=  \prod_{n=1}^N  \text{Normal}(y_n \mid \mathbf{x}_n^\top\mathbf{w} + b, \sigma_y^2).\end{align*}The latent variables are the linear model's weights $\mathbf{w}$ andintercept $b$, also known as the bias.Assume $\sigma_w^2,\sigma_b^2$ are known prior variances and $\sigma_y^2$ is aknown likelihood variance. The mean of the likelihood is given by alinear transformation of the inputs $\mathbf{x}_n$.Let's build the model in Edward, fixing $\sigma_w,\sigma_b,\sigma_y=1$.<jupyter_code>X = tf.placeholder(tf.float32, [None, D])
y_ph = tf.placeholder(tf.float32, [None])

w = Normal(loc=tf.zeros(D), scale=tf.ones(D))
b = Normal(loc=tf.zeros(1), scale=tf.ones(1))
y = Normal(loc=ed.dot(X, w) + b, scale=1.0)<jupyter_output><empty_output><jupyter_text>Here, we define a placeholder `X`. During inference, we pass inthe value for this placeholder according to batches of data.To enable training with batches of varying size, we don't fix the number of rows for `X` and `y`. (Alternatively,we could fix it to be the batch size if training and testing with a fixed size.)  InferenceWe now turn to inferring the posterior using variational inference.Define the variational model to be a fully factorized normal acrossthe weights.<jupyter_code>qw = Normal(loc=tf.get_variable("qw/loc", [D]),
            scale=tf.nn.softplus(tf.get_variable("qw/scale", [D])))
qb = Normal(loc=tf.get_variable("qb/loc", [1]),
            scale=tf.nn.softplus(tf.get_variable("qb/scale", [1])))<jupyter_output><empty_output><jupyter_text>Run variational inference with the Kullback-Leibler divergence.We use $5$ latent variable samples for computingblack box stochastic gradients in the algorithm.(For more details, see the[$\text{KL}(q\|p)$ tutorial](http://edwardlib.org/tutorials/klqp).)For batch training, we will iterate over the number of batches andfeed them to the respective placeholder. We set the number ofiterations to be equal to the number of batches times the number ofepochs (full passes over the data set).<jupyter_code>n_batch = int(N / M)
n_epoch = 5

inference = ed.KLqp({w: qw, b: qb}, data={y: y_ph})
inference.initialize(n_iter=n_batch * n_epoch, n_samples=5, scale={y: N / M})
tf.global_variables_initializer().run()

for _ in range(inference.n_iter):
  X_batch, y_batch = next(data)
  info_dict = inference.update({X: X_batch, y_ph: y_batch})
  inference.print_progress(info_dict)<jupyter_output>390/390 [100%] ██████████████████████████████ Elapsed: 4s | Loss: 10365.830<jupyter_text>When initializing inference, note we scale $y$ by $N/M$, so it is as if thealgorithm had seen $N/M$ as many data points per iteration.Algorithmically, this will scale all computation regarding $y$ by$N/M$ such as scaling the log-likelihood in a variational method'sobjective. (Statistically, this avoids inference being dominated by the prior.)The loop construction makes training very flexible. For example, wecan also try running many updates for each batch.<jupyter_code>n_batch = int(N / M)
n_epoch = 1

inference = ed.KLqp({w: qw, b: qb}, data={y: y_ph})
inference.initialize(
    n_iter=n_batch * n_epoch * 10, n_samples=5, scale={y: N / M})
tf.global_variables_initializer().run()

for _ in range(inference.n_iter // 10):
  X_batch, y_batch = next(data)
  for _ in range(10):
    info_dict = inference.update({X: X_batch, y_ph: y_batch})

  inference.print_progress(info_dict)<jupyter_output>770/780 [ 98%] █████████████████████████████  ETA: 0s | Loss: 9589.711<jupyter_text>In general, make sure that the total number of training iterations is specified correctly when initializing `inference`. Otherwise an incorrectnumber of training iterations can have unintended consequences; for example,`ed.KLqp` uses an internal counter to appropriately decay its optimizer's learning rate step size.Note also that the reported `loss` value as we run thealgorithm corresponds to the computed objective given the currentbatch and not the total data set. We can instead have it reportthe loss over the total data set by summing `info_dict['loss']`for each epoch.  CriticismA standard evaluation for regression is to compare prediction accuracy onheld-out "testing" data. We do this by first forming the posterior predictivedistribution.<jupyter_code>y_post = ed.copy(y, {w: qw, b: qb})
# This is equivalent to
# y_post = Normal(loc=ed.dot(X, qw) + qb, scale=tf.ones(N))<jupyter_output><empty_output><jupyter_text>With this we can evaluate various quantities using predictions fromthe model (posterior predictive).<jupyter_code>print("Mean squared error on test data:")
print(ed.evaluate('mean_squared_error', data={X: X_test, y_post: y_test}))

print("Mean absolute error on test data:")
print(ed.evaluate('mean_absolute_error', data={X: X_test, y_post: y_test}))<jupyter_output>Mean squared error on test data:
0.00525331
Mean absolute error on test data:
0.0523023
{'fraction_non_alphanumeric': 0.11977935382190702, 'fraction_numerical': 0.015497767270816917, 'mean_word_length': 5.0532591414944354, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 30, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '228248', 'license': 'Apache-2.0', 'n_tokens_mistral': 2591, 'n_tokens_neox': 2416, 'n_words': 828, 'path': 'notebooks/batch_training.ipynb', 'repo_name': 'zhangyewu/edward', 'size': '7614'}