<jupyter_start><jupyter_text>Intro to Pyro modelsThis is Part I of the time series tutorial https://pyro.ai/timeSee also [Part II](https://pyro.ai/time/part_ii_inference.ipynb) and[Part III](https://pyro.ai/time/part_iii_custom.ipynb).  SetupFirst install Pyro```shpip install pyro-ppl```and import Pyro and PyTorch<jupyter_code>import math
import torch
import pyro
import pyro.distributions as dist
from matplotlib import pyplot
import warnings
warnings.filterwarnings('ignore')

%matplotlib inline
%config InlineBackend.rc = {'figure.facecolor': (1, 1, 1, 1)}
pyro.enable_validation()
print(pyro.__version__)
print(torch.__version__)<jupyter_output>0.4.1
1.2.0<jupyter_text>The Pyro language has two basic primitives```pyx = pyro.sample(“x”, Bernoulli(0.5))assert isinstance(x, torch.Tensor)pyro.sample(“data”, Normal(0., 1.),            obs=data)theta = pyro.param(“theta”, torch.ones(100),                   constraint=positive)```The first `sample()` generates a random value and records it in the Pyro runtime.The second `sample()` conditions on observed data.The final `param()` statement declares a learnable parameter.  Pyro models are Python functions构建模型，得到观测数据。Here is a Pyro model for Poisson regression<jupyter_code>from graphviz import Source
Source('Digraph {rankdir=LR; slope, intercept -> rate -> counts}')
def model(counts):
    slope = pyro.sample("slope", dist.Normal(0, 0.1))
    intercept = pyro.sample("intercept", dist.Normal(math.log(10), 1))
    for t in range(len(counts)):
        rate = torch.exp(intercept + slope * t)
        counts[t] = pyro.sample("count_{}".format(t), dist.Poisson(rate),
                                obs=counts[t])
    return slope, intercept, counts

model([None] * 4)<jupyter_output><empty_output><jupyter_text>Note this has two sample statements and an observe statement. There are no learnable parameters.  模型可以生成数据Pyro models can generate dataRunning a Pyro model will generate a sample from the prior.<jupyter_code>pyro.set_rng_seed(0)

# We pass counts = [None, ..., None] to indicate time duration.
true_slope, true_intercept, true_counts = model([None] * 10)
print("true_counts = {}".format(torch.stack(true_counts)))

pyplot.figure(figsize=(10, 6), dpi=300)
pyplot.plot([c.item() for c in true_counts]);<jupyter_output>true_counts = tensor([ 9., 10., 10., 12., 16., 13., 20., 16., 34., 29.])<jupyter_text>参数后验分布**We can guess parameters of the model from data**We just saw one way to use a Pyro model -- to call it via `model()`.Another way to use a model is to pass the model to an inference algorithm and let the algorithm guess what the model is doing based on observed data (here `true_counts`). This way of using a model is a _nonstandard interpretation_ of the model.<jupyter_code>%%time
# This uses Pyro inference algorithms which we'll discuss later in Part II.
from pyro.infer.autoguide import AutoDelta
from pyro.infer import SVI, Trace_ELBO
from pyro.optim import Adam

guide = AutoDelta(model)
svi = SVI(model, guide, Adam({"lr": 0.1}), Trace_ELBO())
for i in range(101):
    loss = svi.step(true_counts)  # true_counts is passed as argument to model()
    if i % 10 == 0:
        print("loss = {}".format(loss))
print("true_slope = {}".format(true_slope))
print("true_intercept = {}".format(true_intercept))
guess = guide()
print("guess = {}".format(guess))<jupyter_output>true_slope = 0.15409961342811584
true_intercept = 2.0091562271118164
guess = {'slope': tensor(0.1365, grad_fn=<ExpandBackward>), 'intercept': tensor(2.1379, grad_fn=<ExpandBackward>)}<jupyter_text>The `guide` is also a Pyro modelNote that the `guide` is also a Pyro model (created by `AutoDelta`). The guide has `pyro.param` statements that declare tunable parameters. We'll write custom guides later in [Part III](https://pyro.ai/time/part_iii_custom.ipynb).  We then predict future data有了 guide 之后，我们可以进行预测。We've now seen two ways to use a model: - to generate data from the prior, and- to learn parameters from data. A third way to use a Pyro model is to **predict new observed data by guiding the model**(背后的数学是什么样子？). This uses two of Pyro's effects:- `trace` records guesses made by the guide, and- `replay` conditions the model on those guesses, allowing the model to generate conditional samples.We'll wrap this idiom as a `forecast()` method:<jupyter_code>from graphviz import Source
Source('Digraph {rankdir=LR; slope, intercept -> rate -> counts}')
from pyro import poutine

def forecast(forecast_steps=10):
    counts = true_counts + [None] * forecast_steps  # observed data + blanks to fill in
    guide_trace = poutine.trace(guide).get_trace(counts)
    _, _, counts = poutine.replay(model, guide_trace)(counts)
    return counts

out = forecast(3)
print('observed trace', [x.item() for x in true_counts])
print('trace', [x.item() for x in out])<jupyter_output>observed trace [9.0, 10.0, 10.0, 12.0, 16.0, 13.0, 20.0, 16.0, 34.0, 29.0]
trace [9.0, 10.0, 10.0, 12.0, 16.0, 13.0, 20.0, 16.0, 34.0, 29.0, 39.0, 44.0, 48.0]<jupyter_text>Pyro predicts by drawing samplesWe can now call `forecast()` multiple times to generate samples.<jupyter_code>num_samples = 20
pyplot.figure(figsize=(10, 6), dpi=300)
for _ in range(num_samples):
    full_counts = forecast(10)
    \
    forecast_counts = full_counts[len(true_counts):]
    pyplot.plot([c.item() for c in full_counts], "r",
                label=None if _ else "forecast", alpha=0.3)
pyplot.plot([c.item() for c in true_counts], "k-", label="truth")
pyplot.legend();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12701868989294138, 'fraction_numerical': 0.033932135728542916, 'mean_word_length': 4.796004206098844, 'pattern_counts': {'":': 1, '<': 23, '<?xml version=': 0, '>': 27, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '540879', 'license': 'Apache-2.0', 'n_tokens_mistral': 2004, 'n_tokens_neox': 1842, 'n_words': 575, 'path': '2019-08-time-series/part_i_models.ipynb', 'repo_name': 'HeyangGong/sandbox', 'size': '5511'}