<jupyter_start><jupyter_text>Bayesian regression analysis in lmfitIn this example we will perform a Bayesian regression analysis in the `lmfit` package. You'll need the `emcee` and `corner` packages installed. Both of these are available on PyPI. We start off with the requisite imports.<jupyter_code>%matplotlib inline
import numpy as np
import lmfit
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Now lets create the data we'll then fit.<jupyter_code>x = np.linspace(1, 10, 250)
np.random.seed(0)
y = 3.0*np.exp(-x/2) - 5.0*np.exp(-(x-0.1) / 10.) + 0.1*np.random.randn(len(x))
plt.plot(x, y)<jupyter_output><empty_output><jupyter_text>We'll need a `Parameters` object to store the Parameters of our model.<jupyter_code>p = lmfit.Parameters()
p.add_many(('a1', 4), ('a2', 4), ('t1', 3), ('t2', 3., True))

def residual(p):
    v = p.valuesdict()
    return v['a1']*np.exp(-x/v['t1']) + v['a2']*np.exp(-(x-0.1) / v['t2']) - y<jupyter_output><empty_output><jupyter_text>We'll do a straightforward least-squares fit first, using the *Nelder-Mead* method.<jupyter_code>mi = lmfit.minimize(residual, p, method='Nelder', nan_policy='omit')
lmfit.printfuncs.report_fit(mi.params, min_correl=0.5)
plt.plot(x, y)
plt.plot(x, residual(mi.params) + y, 'r')<jupyter_output><empty_output><jupyter_text>The above analysis ignored the uncertainty on each datapoint. So we'll add in a noise Parameter to address that.<jupyter_code># add a noise parameter
mi.params.add('noise', value=1, min=0.001, max=2)<jupyter_output><empty_output><jupyter_text>For the Bayesian analysis we have to calculate the log-posterior probability for the system. The log-posterior probability is the sum of the log-prior and log-likelihood probabilities. The log-prior encodes prior beliefs about the system. With the `lmfit.emcee` method the log-prior probability is assumed to be 0 if a `Parameter` is within its bounds. If it's outside the bounds the log-prior probability is `-np.inf`, i.e. impossible. This is known as a uniform prior.The log-likelihood function is the probability of the data given the model. The following function calculates the log-likelihood. Note how the data uncertainties (http://dan.iel.fm/emcee/current/user/line/) are included in the log-likelihood.Note, if you want a non-uniform prior, then you should include the log-prior and log-likelihood in a single function.<jupyter_code># This is the log-likelihood probability for the sampling. We're going to estimate the
# size of the uncertainties on the data as well.
def lnprob(p):
    noise = p['noise']
    return -0.5 * np.sum((residual(p) / noise)**2 + np.log(2 * np.pi * noise**2))<jupyter_output><empty_output><jupyter_text>With `lmfit.emcee` we first have to create a `lmfit.Minimizer` object. We're going to initialise it with the best fit from above.Then we do the sampling. We are going to burn in (discard) the first 300 steps out of a total of 1000. We are then going to thin, which reduces the amount of autocorrelation in the chain. Thinning by 20 means that 1 in 20 samples is kept.<jupyter_code>mini = lmfit.Minimizer(lnprob, mi.params)
res = mini.emcee(burn=300, steps=1000, thin=20, params=mi.params)<jupyter_output><empty_output><jupyter_text>Now lets plot the outcome with the `corner` package. Note the banana shapes, they arise out of non-normality for the parameter probability distribution. The elliptical nature of several parameter pairs indicates a high degree of correlation. The correlation parameter is returned as an attribute of the `res` OptimizeResult.<jupyter_code>import corner
a=corner.corner(res.flatchain, labels=res.var_names, truths=list(res.params.valuesdict().values()))<jupyter_output><empty_output><jupyter_text>Here are the output parameters, the correlations are also worked out. The `OptimizeResult` contains `chain` and `lnprob` attributes which contain the samples and corresponding log-probability. In addition, the Minimizer will then also contain a `sampler` attribute (a `emcee.EnsembleSampler` object) which can be used for other purposes.<jupyter_code>print("median of posterior probability distribution")
print('--------------------------------------------')
lmfit.report_fit(res.params)
# find the maximum likelihood solution
highest_prob = np.argmax(res.lnprob)
hp_loc = np.unravel_index(highest_prob, res.lnprob.shape)
mle_soln = res.chain[hp_loc]
for i, par in enumerate(p):
    p[par].value = mle_soln[i]
print("\nMaximum likelihood Estimation")
print('-----------------------------')
print(p)
plt.plot(x, y)
plt.plot(x, residual(mi.params) + y, 'r', label='Nelder-Mead')
plt.plot(x, residual(res.params) + y, 'black', label='emcee')
plt.legend()<jupyter_output><empty_output><jupyter_text>Finally, let's calculate some uncertainty intervals.<jupyter_code>quantiles = np.percentile(res.flatchain['t1'], [2.28, 15.9, 50, 84.2, 97.7])
print("1 sigma spread", 0.5 * (quantiles[3] - quantiles[1]))
print("2 sigma spread", 0.5 * (quantiles[4] - quantiles[0]))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13110181311018132, 'fraction_numerical': 0.018330344690177327, 'mean_word_length': 5.144430844553243, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '614251', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1729, 'n_tokens_neox': 1607, 'n_words': 533, 'path': 'examples/lmfit_emcee_example.ipynb', 'repo_name': 'schachmett/lmfit-py', 'size': '5019'}