<jupyter_start><jupyter_text>Think BayesCopyright 2018 <NAME>MIT License: https://opensource.org/licenses/MIT<jupyter_code># Configure Jupyter so figures appear in the notebook
%matplotlib inline

# Configure Jupyter to display the assigned value after an assignment
%config InteractiveShell.ast_node_interactivity='last_expr_or_assign'

import numpy as np

# import classes from thinkbayes2
from thinkbayes2 import Pmf, Cdf, Suite, Joint

import thinkplot<jupyter_output><empty_output><jupyter_text>Bayesian regressionThis notebook presents a simple example of Bayesian regression using sythetic data DataSuppose there is a linear relationship between `x` and `y` with slope 2 and intercept 1, but the measurements of `y` are noisy; specifically, the noise is Gaussian with mean 0 and `sigma = 0.3`.<jupyter_code>slope = 2
inter = 1
sigma = 0.3
xs = np.linspace(0, 1, 6)
ys = inter + slope * xs + np.random.normal(0, sigma, len(xs))
thinkplot.plot(xs, ys)
thinkplot.decorate(xlabel='x',
                   ylabel='y')<jupyter_output><empty_output><jupyter_text>Grid algorithmWe can solve the problem first using a grid algorithm, with uniform priors for slope, intercept, and sigma.As an exercise, fill in this likelihood function, then test it using the code below.Your results will depend on the random data you generated, but in general you should find that the posterior marginal distributions peak near the actual parameters.<jupyter_code>from scipy.stats import norm

class Regress(Suite, Joint):
    
    def Likelihood(self, data, hypo):
        """
        
        data: x, y
        hypo: slope, inter, sigma
        """
        return 1
# Solution

from scipy.stats import norm

class Regress(Suite, Joint):
    
    def Likelihood(self, data, hypo):
        """
        
        data: x, y
        hypo: slope, inter, sigma
        """
        x, y = data
        slope, inter, sigma = hypo
        
        yfit = inter + slope * x
        error = yfit - y
        like = norm(0, sigma).pdf(error)
        return like
params = np.linspace(-4, 4, 21)
sigmas = np.linspace(0.1, 2, 20)
from itertools import product
hypos = product(params, params, sigmas)
suite = Regress(hypos);
for data in zip(xs, ys):
    suite.Update(data)
thinkplot.Pdf(suite.Marginal(0))
thinkplot.decorate(xlabel='Slope',
                   ylabel='PMF',
                   title='Posterior marginal distribution')
thinkplot.Pdf(suite.Marginal(1))
thinkplot.decorate(xlabel='Intercept',
                   ylabel='PMF',
                   title='Posterior marginal distribution')
thinkplot.Pdf(suite.Marginal(2))
thinkplot.decorate(xlabel='Sigma',
                   ylabel='PMF',
                   title='Posterior marginal distribution')<jupyter_output><empty_output><jupyter_text>MCMCImplement this model using MCMC.  As a starting place, you can use this example from [Computational Statistics in Python](http://people.duke.edu/~ccc14/sta-663-2016/16C_PyMC3.htmlLinear-regression).You also have the option of using the GLM module, [described here](https://docs.pymc.io/notebooks/GLM-linear.html).<jupyter_code>import pymc3 as pm
pm.GLM
thinkplot.plot(xs, ys)
thinkplot.decorate(xlabel='x',
                   ylabel='y')
import pymc3 as pm

with pm.Model() as model:
    """Fill this in"""
# Solution


with pm.Model() as model:
    slope = pm.Uniform('slope', -4, 4)
    inter = pm.Uniform('inter', -4, 4)
    sigma = pm.Uniform('sigma', 0, 2)

    y_est = slope*xs + inter
    y = pm.Normal('y', mu=y_est, sd=sigma, observed=ys)
    trace = pm.sample_prior_predictive(100)
# Solution

for y_prior in trace['y']:
    thinkplot.plot(xs, y_prior, color='gray', linewidth=0.5)
    
thinkplot.decorate(xlabel='x',
                   ylabel='y')
# Solution

with pm.Model() as model:
    slope = pm.Uniform('slope', -4, 4)
    inter = pm.Uniform('inter', -4, 4)
    sigma = pm.Uniform('sigma', 0, 2)

    y_est = slope*xs + inter
    y = pm.Normal('y', mu=y_est, sd=sigma, observed=ys)
    trace = pm.sample(1000, tune=2000)
# Solution

pm.traceplot(trace);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11242022582228768, 'fraction_numerical': 0.017673048600883652, 'mean_word_length': 3.3959007551240563, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 18, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '563490', 'license': 'MIT', 'n_tokens_mistral': 1381, 'n_tokens_neox': 1298, 'n_words': 383, 'path': 'solutions/regress_soln.ipynb', 'repo_name': 'chwebster/ThinkBayes2', 'size': '4083'}