<jupyter_start><jupyter_text>Symbulate Documentation Methods for Common Discrete and Continuous Distributions  Symbulate can function as an analytical distribution calculator to calculate theoretical probabilities, expected values, and quantiles for common [discrete](common_discrete.html) and [continuous](common_continuous.html) distributions.  1. [**Probability density function (pdf)**](pdf)1. [**Cumulative distribution function (cdf)**](cdf)1. [**Expected value (mean)**](mean)1. [**Variance (var) and standard deviation (sd)**](sd)1. [**Quantile**](quantile)1. [**Median**](median)  Be sure to import Symbulate using the following commands.<jupyter_code>from symbulate import *
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Probability density function (pdf) The `.pdf()` method evaluates the probability density function of named distributions.  If $f$ is the pdf of a named continuous `Distribution`, calling `Distribution.pdf(x)` returns the value $f(x)$.*Example.* The Normal(0, 1) pdf evaluated at 0 is $f(0)=1/\sqrt{2\pi}$.<jupyter_code>Normal(mean=0, sd=1).pdf(0)<jupyter_output><empty_output><jupyter_text>For discrete random variables, the probability density function is also known as the probability mass function (pmf).  If $p$ is the pmf of a named discrete `Distribution`, calling `Distribution.pdf(x)` returns the value $p(x)$.*Example*. The Binomial(5, 0.4) pmf evaluated at 3, which is $\binom{5}{3}0.4^3 0.6^2$.<jupyter_code>Binomial(n=5, p=0.4).pdf(3)<jupyter_output><empty_output><jupyter_text>Cumulative distribution function (cdf) Recall that if $X$ is a random variable defined on a probability space with probability measure $P$, then its **cumulative distribution function**, $F:\mathbb{R}\mapsto[0,1]$, is defined by $F(x) = P(X\le x)$.  The `.cdf()` method evaluates the cumulative distribution function of named distributions.  If $F$ is the cdf of a named `Distribution`, calling `Distribution.cdf(x)` returns the value $F(x)$.*Example.* The Normal(0, 1) cdf evaluated at 2 is $\int_{-\infty}^2 \frac{1}{\sqrt{2\pi}} e^{-z^2/2}\, dz$.<jupyter_code>Normal(mean=0, sd=1).cdf(2)<jupyter_output><empty_output><jupyter_text>*Example*. The Binomial(5, 0.4) cdf evaluated at 3 is $\sum_{x=0}^3\binom{5}{x}0.4^x 0.6^{5-x}$.<jupyter_code>Binomial(n=5, p = 0.4).cdf(3)<jupyter_output><empty_output><jupyter_text>Expected value (mean) For a named `Distribution`, calling `Distribution.mean()` returns its [expected value (mean)](rv.htmlmean).*Example.* The expected value of a Gamma distribution with shape parameter $\alpha$ and rate parameter $\lambda$ is $\alpha/\lambda$.<jupyter_code>Gamma(shape=10, rate=2).mean()<jupyter_output><empty_output><jupyter_text>*Example*. The expected value of a Binomial($n$, $p$) distribution is $np$.<jupyter_code>Binomial(n=5, p=0.4).mean()<jupyter_output><empty_output><jupyter_text>There are some technicalities that can be encountered regarding whether an expected value is defined or finite.  If $E(X)$ is defined, it can be written as $E(X) = E(\max(X,0)) - E(-\min(X, 0))$.The expected value of a random variable is finite only if both $E(\max(X,0))$ and $E(-\min(X, 0))$ are finite.It is possible to have infinite expected value, if one of $E(\max(X,0))$ and $E(-\min(X, 0))$ is infinite and the other is finite.*Example.*  The expected value of an $F$ distribution with $d_2$ degrees of freedom in the denominator is infinite if $d_2\le 2$.<jupyter_code>F(dfN=10, dfD=2).mean()<jupyter_output><empty_output><jupyter_text>The expected value of a random variable $X$ is undefined if both $E(\max(X, 0))$ and $E(-\min(X, 0))$ are infinite. *Example.* The expected value of a Cauchy distribution is undefined.<jupyter_code>Cauchy(loc=0, scale=1).mean()<jupyter_output><empty_output><jupyter_text>Variance (var) and standard deviation (sd) For a named `Distribution`, calling `Distribution.var()` returns its [variance](rv.htmlvar) and `Distribution.sd()` returns its [standard deviation](rv.htmlvar).*Example.* The variance of a Gamma distribution with shape parameter $\alpha$ and rate parameter $\lambda$ is $\alpha/\lambda^2$.<jupyter_code>Gamma(shape=10, rate=2).var()<jupyter_output><empty_output><jupyter_text>*Example*. The standard deviation of a Binomial($n$, $p$) distribution is $\sqrt{np(1-p)}$.<jupyter_code>Binomial(n=5, p=0.4).sd()<jupyter_output><empty_output><jupyter_text>Similar to the expected value, technicalities can be encountered regarding whether a variance is defined and finite.*Example.*  For a Student's t distribution, the variance is undefined for 1 degree of freedom, infinite for 2 degrees of freedom, and finite for at least 3 degrees of freedom.<jupyter_code>StudentT(df=1).var()
StudentT(df=2).var()
StudentT(df=3).var()<jupyter_output><empty_output><jupyter_text>Quantile For a named `Distribution`, calling `Distribution.quantile(p)` returns the $p$th quantile $Q(p)$.Roughly, the value $x$ is the $p$th percentile (or quantile) of a distribution of a variable if $p$ percent of values of thevariable are less than or equal to $x$.  For example, saying that 630 is the 90th percentile of SAT Math scores means that 90% of scores are 630 or less.The cumulative distribution function (cdf) of a RV fills in the blank in the following: $x$ is the [blank] percentile.  On the other hand, the quantile function &mdash; essentially the inverse cdf &mdash; fills in the blank in the following: the $p$th percentile is [blank].If a cdf $F$ is continuous and strictly increasing on its the range of possible values, then its inverse $F^{-1}:[0,1]\mapsto\mathbb{R}$ exists and is called the quantile (or percentile) function: $Q(p) = F^{-1}(p)$ for $p\in[0, 1]$.  That is, if $x^*=Q(p)$ then $F(x^*)=p$. *Example.* Scores on the SAT Math exam have an approximate Normal distribution with mean 500 and standard deviation 100.  The 90th percentile of scores is about 630, meaning that about 90% of scores are 630 or less.<jupyter_code>Normal(mean=500, sd=100).quantile(0.9)
Normal(mean=500, sd=100).cdf(628.155)<jupyter_output><empty_output><jupyter_text>For many continuous distributions, the quantile function can be defined as the inverse cdf.  However, in some cases the inverse cdf is not well defined.  This is true in particular for discrete distributions, since the cdf of a discrete distribution is a step function.  For example, for a Binomial(5, 0.5) distribution, $F(x) = 0.5$ for $2\le x <3$, so there is not a unique value $x^*$ which satisfies $F(x^*)=0.5$.  Also, $F(3) = 0.8125$ there is no value $x^*$ for which $F(x^*) = p$ for $0.5 < p < 0.8125$.<jupyter_code>Binomial(5, 0.5).cdf(2)
Binomial(5, 0.5).cdf(2.3)
Binomial(5, 0.5).cdf(3)<jupyter_output><empty_output><jupyter_text>To account for such situations in which the inverse cdf is not well defined, the **quantile (or percentile) function** $Q:[0,1]\mapsto\mathbb{R}$ is defined as$$Q(p) = \inf\{x: F(x) \ge p\}, \qquad \text{for } 0\le p \le 1.$$When $F^{-1}$ exists then $Q=F^{-1}$.*Example*.  For a Binomial(5, 0.5) distribution, $F(x) = 0.5$ for $2\le x <3$, and $F(x)<0.5$ for $x < 2$, so $Q(0.5) = 2$.   Also, since $F(3)=0.8125$ then $Q(p) = 3$ for $0.5 < p \le 0.8125$<jupyter_code>Binomial(n=5, p=0.5).quantile(0.5)
Binomial(n=5, p=0.5).quantile(0.6)
Binomial(n=5, p=0.5).quantile(0.75)
Binomial(n=5, p=0.5).quantile(0.8126)<jupyter_output><empty_output><jupyter_text>Median Roughly, the median is the 50th percentile of a distribution.  If a distribution has quantile function $Q$, then $Q(0.5)$ is the **median**.  For a named `Distribution`, calling `Distribution.median()` returns its median.*Example.* For a Normal distribution, the median and the mean are equal.<jupyter_code>Normal(mean=500, sd=100).median()<jupyter_output><empty_output><jupyter_text>*Example.* As discussed above, the median of a Binomial(5, 0.5) distribution is 0.5.  For a Binomial($n$, $p$) distribution if the mean $np$ is a whole number, then the median is also equal to $np$.<jupyter_code>Binomial(5, 0.5).median()
Binomial(1000, 0.01).median()<jupyter_output><empty_output><jupyter_text>*Example.* The mean of a Cauchy distribution is undefined, but the median is equal to its location parameter.<jupyter_code>Cauchy(loc=0, scale=1).median()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14341132258455572, 'fraction_numerical': 0.03236755970420657, 'mean_word_length': 5.921140939597316, 'pattern_counts': {'":': 0, '<': 80, '<?xml version=': 0, '>': 73, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '192540', 'license': 'MIT', 'n_tokens_mistral': 2926, 'n_tokens_neox': 2688, 'n_words': 928, 'path': 'docs/common_general_comments.ipynb', 'repo_name': 'ksachdeva/symbulate', 'size': '8249'}