<jupyter_start><jupyter_text>Random Signals*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [<EMAIL>](mailto:<EMAIL>).*  Cumulative Distribution FunctionsA random process can be characterized by the statistical properties of its amplitude values. [Cumulative distribution functions](https://en.wikipedia.org/wiki/Cumulative_distribution_function) (CDFs) are one possibility to do so.  Univariate Cumulative Distribution FunctionThe univariate CDF $P_x(\theta, k)$ of a continuous-amplitude real-valued random signal $x[k]$ is defined as\begin{equation}P_x(\theta, k) := \Pr \{ x[k] \leq \theta\}\end{equation}where $\Pr \{ \cdot \}$ denotes the probability that the given condition holds. The univariate CDF quantifies the probability that for the entire ensemble and for a fixed time index $k$ the amplitude $x[k]$ is smaller or equal to $\theta$. The term '*univariate*' reflects the fact that only one random process is considered.The CDF shows the following properties which can be concluded directly from its definition\begin{equation}\lim_{\theta \to -\infty} P_x(\theta, k) = 0\end{equation}and\begin{equation}\lim_{\theta \to \infty} P_x(\theta, k) = 1\end{equation}The former property results from the fact that all amplitude values $x[k]$ are larger than $- \infty$, the latter from the fact that all amplitude values lie within $- \infty$ and $\infty$. The univariate CDF $P_x(\theta, k)$ is furthermore a non-decreasing function.The probability that $\theta_1 < x[k] \leq \theta_2$ is given as\begin{equation}\Pr \{\theta_1 < x[k] \leq \theta_2\} = P_x(\theta_2, k) - P_x(\theta_1, k)\end{equation}Hence, the probability that a continuous-amplitude random signal takes a specific value $x[k]=\theta$ is zero when calculated by means of the CDF. This motivates the definition of probability density functions introduced later.  Bivariate Cumulative Distribution FunctionThe statistical dependencies between two signals are frequently of interest in statistical signal processing. The bivariate or joint CDF $P_{xy}(\theta_x, \theta_y, k_x, k_y)$ of two continuous-amplitude real-valued random signals $x[k]$ and $y[k]$ is defined as\begin{equation}P_{xy}(\theta_x, \theta_y, k_x, k_y) := \Pr \{ x[k_x] \leq \theta_x  \wedge y[k_y] \leq \theta_y  \}\end{equation}The joint CDF quantifies the probability for the entire ensemble of sample functions that for a fixed $k_x$ the amplitude value $x[k_x]$ is smaller or equal to $\theta_x$ and that for a fixed $k_y$ the amplitude value $y[k_y]$ is smaller or equal to $\theta_y$. The term '*bivariate*' reflects the fact that two random processes are considered. The bivariate CDF can also be used to characterize the statistical properties of one random signal $x[k]$ at two different time-instants $k_x$ and $k_y$ by setting $y[k] = x[k]$\begin{equation}P_{xx}(\theta_1, \theta_2, k_1, k_2) := \Pr \{ x[k_1] \leq \theta_1  \wedge y[k_2] \leq \theta_2  \}\end{equation}The definition of the bivariate CDF can be extended straightforward to the case of more than two random variables. The resulting CDF is termed as multivariate CDF.  Probability Density Functions[Probability density functions](https://en.wikipedia.org/wiki/Probability_density_function) (PDFs) describe the probability for one or multiple random signals to take on a specific value. Again the univariate case is discussed first.  Univariate Probability Density FunctionThe univariate PDF $p_x(\theta, k)$ of a continuous-amplitude real-valued random signal $x[k]$ is defined as the derivative of the univariate CDF\begin{equation}p_x(\theta, k) = \frac{\partial}{\partial \theta} P_x(\theta, k)\end{equation}Due to the properties of the CDF and the definition of the PDF, it shows the following properties\begin{equation}p_x(\theta, k) \geq 0\end{equation}and\begin{equation}\int\limits_{-\infty}^{\infty} p_x(\theta, k) \, \mathrm{d}\theta = P_x(\infty, k) = 1\end{equation}The univariate PDF has only positive values and the area below the PDF is equal to one.Due to the definition of the PDF as derivative of the CDF, the CDF can be computed from the PDF by integration\begin{equation}P_x(\theta, k) = \int\limits_{-\infty}^{\theta} p_x(\theta, k) \, \mathrm{d}\theta\end{equation}  Example - Estimate of an univariate PDF by the histogramIn the process of calculating a [histogram](https://en.wikipedia.org/wiki/Histogram), the entire range of amplitude values of a random signal is split into a series of intervals (bins). For a given random signal the number of samples is counted which fall into one of these intervals. This is repeated for all intervals. The counts are finally normalized with respect to the total number of samples. This process constitutes a numerical estimation of the PDF of a random process.In the following example the histogram of an ensemble of random signals is computed for each time index $k$. The CDF is computed by taking the cumulative sum over the histogram bins. This constitutes a numerical approximation of above integral\begin{equation}\int\limits_{-\infty}^{\theta} p_x(\theta, k) \approx \sum_{i=0}^{N} p_x(\theta_i, k) \, \Delta\theta_i\end{equation}where $p_x(\theta_i, k)$ denotes the $i$-th bin of the PDF and $\Delta\theta_i$ its width.<jupyter_code>%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

K = 32  # number of temporal samples
N = 10000  # number of sample functions
bins = 100  # number of bins for the histogram


# draw sample functions from a random process
np.random.seed(2)
x = np.random.normal(size=(N, K))
x += np.tile(np.cos(2*np.pi/K*np.arange(K)), [N, 1])

# compute the histogram
px = np.zeros((bins, K))
for k in range(K):
    px[:, k], edges = np.histogram(x[:, k], bins=bins, range=(-4,4), density=True)
    
# compute the CDF
Px = np.cumsum(px, axis=0) * 8/bins

# plot the PDF
plt.figure(figsize=(10,6))
plt.pcolor(np.arange(K), edges, px)
plt.title(r'Estimated PDF $\hat{p}_x(\theta, k)$')
plt.xlabel(r'$k$')
plt.ylabel(r'$\theta$')
plt.colorbar()
plt.autoscale(tight=True)

# plot the CDF
plt.figure(figsize=(10,6))
plt.pcolor(np.arange(K), edges, Px, vmin=0, vmax=1)
plt.title(r'Estimated CDF $\hat{P}_x(\theta, k)$')
plt.xlabel(r'$k$')
plt.ylabel(r'$\theta$')
plt.colorbar()
plt.autoscale(tight=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1140034498980712, 'fraction_numerical': 0.006899796142386702, 'mean_word_length': 4.949626865671642, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 7, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '24701', 'license': 'MIT', 'n_tokens_mistral': 2011, 'n_tokens_neox': 1880, 'n_words': 773, 'path': 'random_signals/distributions.ipynb', 'repo_name': 'ganlubbq/digital-signal-processing-lecture', 'size': '6417'}