<jupyter_start><jupyter_text>Statistical testsThis presentation will briefly go over MANOVAs (and also a bit of ANOVAs to make sense of MANOVAs). I have deliberately put in no equations!  Basic theory stuff about ANOVAs and MANOVAs  Hypothesis testingHypothesis testing is a method of statistical inference used to determine possbile probabilities of conflicting hypotheses (usually null and alternative hypothesis). Depending on our data and hypothesis we can use a variety of different tests to calculate the probalitiy of our hypothesis.Broadly two categories of test exist:- Parametric (which makes assumptions regarding the distriubtion of our data)- Non Parametric (which makes less assumptions)  Parametric tests T-tests and ANOVAs.T-tests, ANOVAs and by extension MANOVAs are common hypothesis driven parametric statistical tests.  T-tests and ANOVAs test if two or more means are sampled from the same distribution (i.e the null distribution). The null hypothesis for t-tests and ANOVAs is that the means are sampled from the same distriubtion.ANOVAs are concerned with analysing the variance bewtween and within groups.We won't really dwell on t-tests apart from to say that t-tests come in different flavours.  How do ANOVAs work?ANOVAs are a type of linear regression that uses an F test to then test for significance (basically it runs a regression then uses another test to calculate p values etc).ANOVAs work by running a regression. From the regression an ANOVA then calculates the estimates of variance (called mean squares). An ANOVA calculates the total variance, the within group variance and the between group variance. Once an ANOVA has all that it can then caculate an F statistic, p values degrees of freedom and effect sizes.   ANOVA exampleWe have three groups, a actuely underweight group of individuals with Anorexia Nervosa, a weight-restored group and healthy controls and we want to examine the effect of group on ede-q scores.The null hypothesis is that ede-q scores will not differ by group (all are sampled from the same null distribution) while the alternative hypothesis is that ede-q scores will differ by group (sampled from different distributions).We can then run a one-way ANOVA with group as a catergorical independent variable and ede-q as a continious dependent variable to see if a group difference exists.   MANOVAMANOVA is basically the same as an ANOVA but with one key difference. Instead of having one dependent variable a MANOVA can have two or more. So in our example on the previous slide if we wanted to check if ede-q scores AND AQ10 scores differed by group we would use a MANOVA rather than an ANOVA (we could also do two ANOVAs).Mathematically MANOVAs work by testing a **vector** of two or more means are sampled from the same distribution. The null hypothesis is that the vector of means will be sampled from the same distriubtion, the alternative hypothesis is that the vector of means are taken from different multivariate distriubtions.  When to use a MANOVAWhy use a MANOVA when we could just do multiple ANOVAs?MANOVAs are useful:**1) To explore how independent variables influence some patten of response in the dependent variables.**2) When there are several correlated variables and a need for an overall statistical test with one value instead of performing multiple tests.3) Used when assumptions of sphericity is violated (how equal the variance is, more equal variance = Sphericity)  How MANOVAs work.They work pretty much exactly the same as ANOVAs when running the regression and calculating variance. The difference is all the maths is now in matrix form.The real difference between MANOVAs and ANOVAs is MANOVAs don't have one defintive test to calculate the significance and F statistics but rather a few. 1) Pillai's trace (Conisdered the most powerful and robust). 2) Hotelling-Lawley's trace.3) Wilk's lambda (the first statistic to be used in multivariate analysis).4) Roy's largest root (Often considered the weakest).  An example of a MANOVA in python from (pretty much) start to finishIn this example we have three therapy methods and have collected two measures, one of client satisfacion and one test scores for a wellbeing questionnaire. We want to examine the realtionship between therapy methods and our two measures as well as seeing if there is a difference between therapy method and our two measures<jupyter_code>#Code to read in the data
import pandas as pd 

#Code to do the manova
import statsmodels.multivariate.manova as manova 

#Code to plot
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('dark')
df = pd.read_csv('MANOVA_example.csv')
groups = df.groupby('Method')
groupa = groups.get_group(1)
groupb = groups.get_group(2)
groupc = groups.get_group(3)
des = pd.concat([groupa[['Test','Satisfaction']].describe().T, groupb[['Test','Satisfaction']].describe().T, groupc[['Test','Satisfaction']].describe().T], 
                 keys=['Method 1', 'Method 2', 'Method 3'])
print(df.head(),'\n')
print(df.dtypes, '\n')
print('Number of null values:', sum(df.isnull().sum()))
print(des)
fig, ax = plt.subplots(1,2, figsize=(16,8))
fig.suptitle('Data exploration')

sns.boxplot(ax=ax[0], data=df, x='Method', y='Satisfaction', hue=df['Method'].tolist(), linewidth=1)
sns.stripplot(ax=ax[0], data=df, x='Method', y='Satisfaction', color=".15")

sns.boxplot(ax=ax[1], data=df, x='Method', y='Test', hue=df['Method'].tolist(), linewidth=1)
sns.stripplot(ax=ax[1], data=df, x='Method', y='Test', color=".15")
sns.scatterplot(data=df, x='Satisfaction', y='Test', hue=df['Method'].tolist())
fig, ax = plt.subplots(1,4, figsize=(25,6))
sns.regplot(ax=ax[0], data=df, x='Satisfaction', y='Test', ci=False)
sns.regplot(ax=ax[1], data=groupa, x='Satisfaction', y='Test', ci=False)
sns.regplot(ax=ax[2], data=groupb, x='Satisfaction', y='Test', ci=False)
sns.regplot(ax=ax[3], data=groupc, x='Satisfaction', y='Test', ci=False)
from scipy.stats import pearsonr

rval_groupa, pval_group_a = pearsonr(groupa['Test'], groupa['Satisfaction'])
rval_groupb, pval_group_b = pearsonr(groupb['Test'], groupb['Satisfaction'])
rval_groupc, pval_group_c = pearsonr(groupc['Test'], groupc['Satisfaction'])

print(f'Therapy group one, Test and Satisfaction correlation: pval {pval_group_a}, r2 {rval_groupa}')
print(f'Therapy group one, Test and Satisfaction correlation: pval {pval_group_b}, r2 {rval_groupb}')
print(f'Therapy group one, Test and Satisfaction correlation: pval {pval_group_c}, r2 {rval_groupc}')
fit = manova.MANOVA.from_formula('Test + Satisfaction ~ Method', data=df)
fitted_model = fit.mv_test()
print(fitted_model)<jupyter_output>Multivariate linear model
===================================================================
                                                                   
-------------------------------------------------------------------
       Intercept          Value   Num DF  Den DF   F Value   Pr > F
-------------------------------------------------------------------
          Wilks' lambda    0.0002 2.0000 45.0000 131226.2913 0.0000
         Pillai's trace    0.9998 2.0000 45.0000 131226.2913 0.0000
 Hotelling-Lawley trace 5832.2796 2.0000 45.0000 131226.2913 0.0000
    Roy's greatest root 5832.2796 2.0000 45.0000 131226.2913 0.0000
-------------------------------------------------------------------
                                                                   
--------------------------------------------------------------------
            Method          Value   Num DF   Den DF  F Value  Pr > F
-------------------------------------------------------------------[...]<jupyter_text>For fun?? lets test out the ANOVA<jupyter_code>import statsmodels.api as sm
from statsmodels.formula.api import ols

print('ANOVA TABLE for Wellbeing test and Therapy method\n', 
       sm.stats.anova_lm(ols('Method ~ Test', data=df).fit(), typ=1))

print('\nANOVA TABLE for client Satisfaction and Therapy method\n', 
        sm.stats.anova_lm(ols('Method ~ Satisfaction', data=df).fit(), typ=1))
print(ols('Method ~ Test', data=df).fit().summary())<jupyter_output>OLS Regression Results                            
==============================================================================
Dep. Variable:                 Method   R-squared:                       0.050
Model:                            OLS   Adj. R-squared:                  0.029
Method:                 Least Squares   F-statistic:                     2.426
Date:                Thu, 21 Apr 2022   Prob (F-statistic):              0.126
Time:                        17:20:12   Log-Likelihood:                -57.144
No. Observations:                  48   AIC:                             118.3
Df Residuals:                      46   BIC:                             122.0
Df Model:                           1                                         
Covariance Type:            nonrobust                                         
==============================================================================
                 coef    std err          t      P>|[...]
{'fraction_non_alphanumeric': 0.13883078278101948, 'fraction_numerical': 0.02422107233292965, 'mean_word_length': 3.0159151193633953, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '642859', 'license': 'MIT', 'n_tokens_mistral': 2652, 'n_tokens_neox': 2333, 'n_words': 1001, 'path': 'presentations/MANOVA_presentation.ipynb', 'repo_name': 'WMDA/BB_data', 'size': '9083'}