<jupyter_start><jupyter_text>Inverse Transform sampling Rationale**Inverse transform sampling** allows to transform samples from uniform distribution $U$ to any other distribution $D$, given the $CDF$ of $D$.How can we do it?Let's take$$\large T(U) = X$$where:* $U$ is a uniform random variable* $T$ is some kind of a transformation* $X$ is the target random variable (let's use **exponential** distribution as an example)Now, we said that to perform **inverse transformation sampling**, we need a $CDF$.By definition $CDF$ (we'll call it $F_X(x)$ here) is given by: $$\large F_X(x) \triangleq P(X \leq x)$$We said before that to get $X$, we'll apply certain transformation $T$ to a uniform random variable.We can then say, that:$$\large P(X \leq x) = P(T(U) \leq x)$$Now, let's apply an ibnverse of $T$ to the both sides of the inequality:$$\large = P(U \leq T^{-1}(x))$$Uniform distribution has a nice property that it's $CDF$ at any given point $x$ is equal to the value of $x$.Therefore, we can say that:$$\large = T^{-1}(x)$$and conclude that:$$\large F_X(x) = T^{-1}(x)$$ ConclusionWe demonstrated how to sample from any density $D$ using a sample from a uniform distribution and an inverse of $CDF$ od $D$. Now, let's apply it in practice!  CodeLet's see how to apply this in Python. We'll use **exponential distribution** as an example.<jupyter_code># Define params
SAMPLE_SIZE = 100000
N_BINS = np.sqrt(SAMPLE_SIZE).astype('int') // 2
LAMBDA = 8<jupyter_output><empty_output><jupyter_text>Let's instantiate distributions.We will instantiate an exponential distribution expicitly for comparison purposes.___________Note that **`scipy.stats`** has a slightly **different parametrization** of exponential than the populuar $\lambda$ parametrization. In the [documentation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.expon.html), we read:*A common parameterization for expon is in terms of the rate parameter lambda, such that pdf = lambda * exp(-lambda * x). This parameterization corresponds to using scale = 1 / lambda.*____________Therefore, we're going to use **`scale=1/LAMBDA`** to parametrize our test **exponential distribution**.<jupyter_code># Instantiate U(0, 1)
unif = stats.uniform(0, 1)

# Instantiate Exp(8) for comparison purposes
exp = stats.expon(loc=0, scale=1/LAMBDA)<jupyter_output><empty_output><jupyter_text>Now, we need to define the inverse transformation $T^{-1}(x)$ that will allow us to translate between uniform and exponential samples.The $CDF$ of exponential distribution is defined as:$$\large\begin{equation}  F_X(x) \triangleq    \begin{cases}      1 - e^{-\lambda x} \ \text{  for  }\ x \geq 0\\      0 \ \ \ \ \ \ \ \ \ \ \ \ \ \  \text{for }\ x<0 \\    \end{cases}       \end{equation}$$Let's take the inverse of this function (solve for $x$):$$\large y = 1 - e^{-\lambda x}$$* subtract $1$ from both sides:$$\large 1 - y = - e^{-\lambda x}$$* take $ln$ of both sides:$$\large ln(1 - y) = \lambda x$$* divide both sides by $\lambda$:$$\large x = \frac{ln(1 - y)}{\lambda}$$**Et voilà!** 🎉🎉🎉 We've got it! 💪🏼Let's translate it to Python code:<jupyter_code># Define 
def transform_to_exp(x, lmbd):
    
    """Transoforms a uniform sample into an exponential sample"""
    
    return -np.log(1 - x) / lmbd<jupyter_output><empty_output><jupyter_text>Take samples:<jupyter_code># Sample from uniform
sample_unif = unif.rvs(SAMPLE_SIZE)

# Sample from the true exponential
sample_exp = exp.rvs(SAMPLE_SIZE)

# Transform U -> Exp
sample_transform = transform_to_exp(sample_unif, LAMBDA)<jupyter_output><empty_output><jupyter_text>A brief sanity check:<jupyter_code># Sanity check -> U(0, 1)
plt.hist(sample_unif, bins=N_BINS, density=True)
plt.title('Histogarm of $U(0, 1)$')
plt.ylabel('$p(x)$')
plt.xlabel('$x$')
plt.show()<jupyter_output><empty_output><jupyter_text>..and let's compare the resutls:<jupyter_code>plt.hist(sample_exp, bins=N_BINS, density=True, alpha=.5, label='Exponential')
plt.hist(sample_transform, bins=N_BINS, density=True, alpha=.5, label='$T(U)$')
plt.legend()
plt.title('Histogram of exponential and transformed distributions', fontsize=12)
plt.ylabel('$p(x)$')
plt.xlabel('$x$')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14547183265985264, 'fraction_numerical': 0.009270263845971, 'mean_word_length': 5.116279069767442, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 27, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '503556', 'license': 'MIT', 'n_tokens_mistral': 1465, 'n_tokens_neox': 1366, 'n_words': 458, 'path': 'sampling/00 - Inverse transform sampling.ipynb', 'repo_name': 'AlxndrMlk/statistics', 'size': '4207'}