<jupyter_start><jupyter_text>PyMC4 Made Simple for PyMC3 UsersIn this notebook, we will use a simple Bayesian estimation example to learn how to use the PyMC4 syntax.  Simulated DataTo keep things simple, we will use a simple example in which we generate 1,000 data points from a normal distribution with a pre-specified $\mu$ and $\sigma$.<jupyter_code>MU = 8
SIG = 2.2

data = np.random.normal(loc=MU, scale=SIG, size=200)<jupyter_output><empty_output><jupyter_text>The goal of our estimation task is to estimate the true value of $\mu$ and $\sigma$ from the observed data.  Model DefinitionPyMC models are defined as functions with a `@pm.model` decorator on top.To specify random variables as Python objects, we use the `yield` statement. This stems from PyMC4's API design, which uses coroutines underneath the hood. The technical reason is documented in the PyMC4 design guide; what's cool here is that it also serves as a "visual hack" that lets you very quickly identify all of your random variables in a model. We're going to define our model in the code cell below.As a prior for the $\mu$, we will use a relatively flat Normal distribution, and for the $\sigma$ prior, a relatively flat Exponential distribution.Unlike PyMC3's distributions, which used spelled Greek letters as arguments, in PyMC4, we use the standardized "location", "scale", "rate" and "concentration" paradigm used by TensorFlow Probability's distributions, as well as NumPy.<jupyter_code>@pm.model
def model(data):
    mu = yield pm.Normal(loc=0, scale=10, name="mu")
    sig = yield pm.Exponential(rate=0.1, name="sig")
    
    like = yield pm.Normal(loc=mu, scale=sig, observed=data, name="like")
    
    return like<jupyter_output><empty_output><jupyter_text>Sampling from PosteriorSampling from posterior distributions is more or less the same as in PyMC3.We call on the model function, and then pass the result to `pm.sample`.Unlike PyMC3, `pm.sample(model)` now returns both the trace _and_ the computed sampling stats.Give it a moment to sample; as of the alpha version of PyMC4, the progress bar is unavailable because sampling is also delegated to TensorFlow probability.<jupyter_code>estimation_model = model(data)

trace = pm.sample(model(data), num_samples=800)<jupyter_output><empty_output><jupyter_text>The trace returned is an ArviZ's InferenceData object, which is the central data format for ArviZ. InferenceData itself is just a container that maintains references to one or more `xarray.Dataset`. You can check the InferenceData structure specification [here](https://arviz-devs.github.io/arviz/schema/schema.html).  Visualizing Posterior DistributionsVisualizations were completely delegated to `arviz` in PyMC3, and that is the same in PyMC4.<jupyter_code>az.plot_posterior(trace, var_names="model/mu");<jupyter_output><empty_output><jupyter_text>We recovered the true $\mu$!<jupyter_code>az.plot_posterior(trace, var_names="model/__log_sig");<jupyter_output><empty_output><jupyter_text>If we take the exponent of the posterior distribution trace values, we will recover back approximately 2.2 for the $\sigma$ as well.  Prior/Posterior Predictive SamplesDoing prior and posterior predictive sampling is an important part of the modelling workflow. Just like in PyMC3, this functionality has been added to PyMC4.<jupyter_code>draws_prior = pm.sample_prior_predictive(estimation_model)
ax = az.plot_kde(draws_prior.prior_predictive['model/like'], label='prior_predictive');
ax = az.plot_kde(data, label='data', plot_kwargs={'color':'C1'});<jupyter_output><empty_output><jupyter_text>`draws_prior` Is also an InferenceData object as such is ordered into groups, in this case we have a single group `prior_predictive`.<jupyter_code>draws_prior<jupyter_output><empty_output><jupyter_text>We drew 1,000 i.i.d. samples from the prior distributions for each of the random variables.<jupyter_code>draws_prior.prior_predictive<jupyter_output><empty_output><jupyter_text>Let make predictions from the fitted model<jupyter_code>draws_posterior = pm.sample_posterior_predictive(estimation_model, trace, 
                                                 inplace=False)<jupyter_output>WARNING:tensorflow:Note that RandomStandardNormal inside pfor op may not give same output as inside a sequential loop.<jupyter_text>`draws_posterior` Is also an InferenceData object, with the group `posterior_predictive`.<jupyter_code>draws_posterior<jupyter_output><empty_output><jupyter_text>Alternatively, we could have called `pm.sample_posterior_predictive` with the argument `inplace=True` to add the `posterior_predictive` group to `trace` instead of generating a new object.Let's see what we got from sampling from the posterior.<jupyter_code>draws_posterior.posterior_predictive<jupyter_output><empty_output><jupyter_text>We had 10 chains, 800 samples for each observation (1st and 2nd dim) and 1000 observations.To make this clearer, <NAME> (one of the PyMC developers) provided the following explanation:> The shape of the output for a posterior predictive samples is (n_chains, samples_per_chain) + shape_of_rv output>> The shape of the RV in posterior predictive sampling takes the shape of the supplied observed values into account. This is not like what prior predictive sampling does, where it only cares about the shape of the distribution. InferenceData objects can be easily concatenated, so you can have all the relevant data in the same place<jupyter_code>combined_trace = trace + draws_posterior + draws_prior
combined_trace<jupyter_output><empty_output><jupyter_text>This is handy for example when performing posterior predictive checks<jupyter_code>az.plot_ppc(combined_trace);<jupyter_output>/home/osvaldo/anaconda3/lib/python3.7/site-packages/IPython/core/events.py:88: UserWarning: Creating legend with loc="best" can be slow with large amounts of data.
  func(*args, **kwargs)
/home/osvaldo/anaconda3/lib/python3.7/site-packages/IPython/core/pylabtools.py:128: UserWarning: Creating legend with loc="best" can be slow with large amounts of data.
  fig.canvas.print_figure(bytes_io, **kw)
{'fraction_non_alphanumeric': 0.0760137908389427, 'fraction_numerical': 0.01001477589886718, 'mean_word_length': 6.026528258362168, 'pattern_counts': {'":': 0, '<': 52, '<?xml version=': 0, '>': 55, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '95851', 'license': 'Apache-2.0', 'n_tokens_mistral': 1836, 'n_tokens_neox': 1735, 'n_words': 657, 'path': 'notebooks/basic-usage.ipynb', 'repo_name': 'irvifa/pymc4', 'size': '6096'}