<jupyter_start><jupyter_text>Day 18: Random Vectors, Continued---- cleanup- multivariate gaussian  - fully defined by its mean and covariance  - isotonic gaussian  - basic calculations: sums of gaussians- limitations of the multivariate gaussian  - Infinite support  - Symmetry  - Unimodal<jupyter_code>import pandas as pd 
import grama as gr 
import numpy as np

from scipy.stats import lognorm
from plotnine import *

DF = gr.Intention()<jupyter_output><empty_output><jupyter_text>Cleanup---Here are some more basic facts we didn't get to last time:<jupyter_code>from grama.models import make_prlc_rand
md_prlc = make_prlc_rand()<jupyter_output><empty_output><jupyter_text>IndependenceTwo variables $X, Y$ are *independent* if their joint CDF factors:$$F(x, y) = F_X(x) F_Y(y)$$> When you call `gr.cp_copula_independence()`, you are assuming that all variables are mutually independent.For instance, the `prlc` model assumes the component values vary independently, which we can see under `copula:`<jupyter_code>md_prlc<jupyter_output><empty_output><jupyter_text>Covariance$$\text{Var}[X] = \mathbb{E}[(X - \mu_X)(X - \mu_X)]$$$$\text{Cov}[X, Y] = \mathbb{E}[(X - \mu_X)(Y - \mu_Y)]$$Variance is a measure of variation in a single variable $X$.Covariance is a measure of co-variation between two variables $X, Y$.  CorrelationA dimensionless measure of dependence$$\rho_{X, Y} = \frac{\text{Cov}[X,Y]}{\sigma_X \sigma_Y}$$  Important caveat: Correlation is **not** a slope!![](https://upload.wikimedia.org/wikipedia/commons/d/d4/Correlation_examples2.svg)This is a super-important figure. The value above each panel gives the correlation $\rho_{X,Y}$ for that set of data. Note that correlation is not a slope! Rather, it is a measure of how close to linear a trend is. The less variable that trend, the close to $\pm1$ is $\rho$. Also, nonlinear trends can be entirely missed by $\rho$.  Important caveat!A lack of correlation does **not** mean a lack of dependence. The following data have exactly zero correlation, but clearly there's some dependence between the variables:<jupyter_code>df_quad = (
    gr.df_make(x=np.linspace(-1, +1, num=200))
    >> gr.tf_mutate(y=DF.x**2 - 0.5)
)

print(
    df_quad
    >> gr.tf_summarize(rho=gr.corr(DF.x, DF.y))
)

(
    df_quad
    >> ggplot(aes("x", "y"))
    + geom_point()
)<jupyter_output>rho
0  6.245005e-17<jupyter_text>However, independence *does* imply zero covariance:$$\text{Cov}[X, Y] = \int\int (x - \mu_X)(y - \mu_Y) f_{X,Y}(x, y) dx dy = \int\int (x - \mu_X)(y - \mu_Y) f_X(x) f_Y(y) dx dy = \mathbb{E}[X - \mu_X] \mathbb{E}[Y - \mu_Y] = 0$$and hence zero correlation.  Multivariate Gaussian---Here's a "golden rule" that we'll unpack:> "A multivariate gaussian is fully defined by its mean and variance"Let's start from an example of an multivariate gaussian, and point out the general properties from this specific example:<jupyter_code>md_mv = (
    gr.Model("Multivariate Gaussian")
    >> gr.cp_marginals(
        x=dict(dist="norm", loc=0, scale=1),
        y=dict(dist="norm", loc=0, scale=1),
        z=dict(dist="norm", loc=0, scale=1),
    )
    >> gr.cp_copula_gaussian(
        df_corr=gr.df_make(
            var1=["x", "x", "y"],
            var2=["y", "z", "z"],
            corr=[0.0, 0.4, 0.9],
        )
    )
)
md_mv<jupyter_output><empty_output><jupyter_text>Generate some samples and visualize<jupyter_code>(
    md_mv
    >> gr.ev_monte_carlo(n=1e3, df_det="nom", skip=True)
    >> gr.pt_auto()
)<jupyter_output>eval_monte_carlo() is rounding n...
Design runtime estimates unavailable; model has no timing data.
Calling plot_scattermat....<jupyter_text>Note that in the $d=3$ dimensional case, we have- three means $\mu_1, \mu_2, \mu_3$- three standard deviations $\sigma_1, \sigma_2, \sigma_3$- three correlations $\rho_{1,2}, \rho_{1,3}, \rho_{2,3}$ In the $d=4$ dimensional case, we have- four means $\mu_1, \mu_2, \mu_3, \mu_4$- four standard deviations $\sigma_1, \sigma_2, \sigma_3, \sigma_4$- six correlations $\rho_{1,2}, \rho_{1,3}, \rho_{1,4}, \rho_{2,3}, \rho_{2,4}, \rho_{3,4}$  Mean and Variance of the GaussianWe package together the parameters above into a vector mean $\mu$ and variance-covariance matrix $\Sigma$. In the three-dimensional case we have:$$\mu^{\top} = \left(\mu_1, \mu_2, \mu_3\right)$$$$\Sigma = \left(\begin{array}{ccc} \sigma_1^2 & \sigma_1\sigma_2 \rho_{1,2} & \sigma_1\sigma_3 \rho_{1,3} \\ \sigma_1\sigma_2 \rho_{2,1} & \sigma_2^2 & \sigma_2\sigma_3 \rho_{2,3} \\ \sigma_1\sigma_3 \rho_{3,1} & \sigma_2\sigma_3 \rho_{3,2} & \sigma_3^2 \end{array}\right)$$ It's often more convenient to think in terms of the mean $\mu$, standard deviations $\sigma_1, \dots, \sigma_d$, and correlation matrix $R$. We can relate the matrix of standard deviations $D = \text{Diag}[\sigma_1, \dots, \sigma_d]$, correlation matrix $R$, and covariance via$$\Sigma = D R D$$In the 3d case, these are$$R = \left(\begin{array}{ccc} 1 & \rho_{1,2} & \rho_{1,3} \\ \rho_{2,1} & 1 & \rho_{2,3} \\ \rho_{3,1} & \rho_{3,2} & 1 \end{array}\right)$$$$D = \left(\begin{array}{ccc} \sigma_1 & 0 & 0 \\ 0 & \sigma_2 & 0 \\ 0 & 0 & \sigma_3 \end{array}\right)$$  ExamplesMake sure to test:- Isotonic- Uncorrelated, different variances- Correlated, same variances- Correlated, different variances<jupyter_code>(
    gr.Model("Multivariate Gaussian")
    >> gr.cp_marginals(
        x=dict(dist="norm", loc=0, scale=1),
        y=dict(dist="norm", loc=0, scale=1),
    )
    >> gr.cp_copula_gaussian(
        df_corr=gr.df_make(
            var1=["x"],
            var2=["y"],
            corr=[0.0],
        )
    )
    
    >> gr.ev_monte_carlo(n=1e3, df_det="nom", skip=True)
    >> gr.pt_auto()
)<jupyter_output>eval_monte_carlo() is rounding n...
Design runtime estimates unavailable; model has no timing data.
Calling plot_scattermat....
{'fraction_non_alphanumeric': 0.1561963611397185, 'fraction_numerical': 0.024716786817713696, 'mean_word_length': 4.0232758620689655, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 50, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '213974', 'license': 'MIT', 'n_tokens_mistral': 2187, 'n_tokens_neox': 2072, 'n_words': 581, 'path': 'day18.ipynb', 'repo_name': 'zdelrosario/pdo-notes', 'size': '5826'}