<jupyter_start><jupyter_text>In the dataset, a large sample of adult participants from the general community completed a five factor model personality questionnaire, and three self-report measures related to mood disorders: The Mood Disorder Questionnaire (MDQ), the Hypomania Checklist (HCL), and the Quick Inventory of Depressive Symptomatology (QIDS). The latter three measures were scored such that individuals ended up with a positive screen (scored 1), or a negative screen (scored 0) for being considered 'at-risk' for developing a mood disorder. The researchers were interested in the extent to which the personality variables could predict risk for mood disorders in this dataset.<jupyter_code>df = pd.read_csv('data_lab.csv')
df.info()
questionnaires = [
    'OpenessTotal', 
    'ConscientiousnessTotal', 
    'ExtraversionTotal', 
    'AgreeablenessTotal', 
    'NeuroticismTotal'
]
screening_variables = ['MDQ_screen', 'QIDS_screen', 'HCL_screen']<jupyter_output><empty_output><jupyter_text>Q1. Check that the three outcome variables have been labeled correctly 1 = positive screen, 0 = negative screen. In this research scenario, which category should we consider the target group and which should we consider the reference group?<jupyter_code>df[screening_variables].describe()<jupyter_output><empty_output><jupyter_text>Q2. What proportion of the sample have a positive screen for each of the three outcome variables?<jupyter_code>df[screening_variables].sum() / len(df)<jupyter_output><empty_output><jupyter_text>Q3. Is receiving a positive or negative screen on the QIDS associated with gender? What proportion of men and women have a received a positive screen in relation to the total number of males and females, respectively?<jupyter_code>df.groupby(['Gender', 'QIDS_screen']).size()
df.groupby('Gender')['QIDS_screen'].sum() / df.groupby('Gender')['QIDS_screen'].count()<jupyter_output><empty_output><jupyter_text>Q4. What is the odds ratio for being a positive screen for females compared to males?<jupyter_code>(480 * 1963) / (295 * 1762)<jupyter_output><empty_output><jupyter_text>Q5. The research team initially wanted to examine which of the five personality traits significantly relate to the screening outcome on the QIDS. Run a binomial logistic model. First, check that the reference level is set correctly for the outcome variable in the menu (it should say 'negative'). Why does the overall model test have 5df? Does the model fit information in the output suggest this set of predictors provide a reasonable fit to the outcome variable? Provide the key parts of the output below. What is the deviance value for the null model?<jupyter_code>endog = df['QIDS_screen'].values

exog = df[questionnaires].values
exog = (exog - exog.mean()) / exog.std()
exog = sm.add_constant(exog)

logit_mod = sm.Logit(endog, exog)
residuals = logit_mod.fit()

residuals.summary()<jupyter_output>Optimization terminated successfully.
         Current function value: 0.369630
         Iterations 7<jupyter_text>Q6. What does the omnibus likelihood ratio tests table tell us? Why do the statistics have 1 df?<jupyter_code>full_model = sm.Logit(endog, exog)
full_residuals = full_model.fit()

for coef in range(1, 6):
    
    constrained_mod = sm.Logit(endog, np.delete(exog, coef, 1))
    constrained_residuals = constrained_mod.fit()
    
    p = ll_ratio_test(
        constrained_residuals.llf,
        full_residuals.llf,
    )
    print('')
    print(f'Coeff {coef}: {round(p, 3)}')
    print('')<jupyter_output>Optimization terminated successfully.
         Current function value: 0.369630
         Iterations 7
Optimization terminated successfully.
         Current function value: 0.371107
         Iterations 7

Coeff 1: 0.0

Optimization terminated successfully.
         Current function value: 0.373250
         Iterations 7

Coeff 2: 0.0

Optimization terminated successfully.
         Current function value: 0.370145
         Iterations 7

Coeff 3: 0.032

Optimization terminated successfully.
         Current function value: 0.369979
         Iterations 7

Coeff 4: 0.078

Optimization terminated successfully.
         Current function value: 0.442572
         Iterations 6

Coeff 5: 0.0<jupyter_text>Q7: Examine the omnibus likelihood ratio tests for each predictor, and the information in the model coefficients table. What do they suggest in terms of which of the personality variables are best predictor/s of the screening outcome for the QIDS? What should we bear in mind when interpreting these coefficients in this specific dataset? Q8. On the basis of the above results, the researchers decided to run another model where they examined the effect of neuroticism on the outcome, but this time including age and gender as control variables (given the association gender has with the outcome, as shown earlier). Re-run the binomial LR model, with age, gender and neuroticism as predictors. Summarise the key results below. What conclusions would we reach?<jupyter_code>endog = df['QIDS_screen'].values

exog = df[questionnaires + ['Age']].values
exog = (exog - exog.mean()) / exog.std()
exog = np.append(exog, df['Gender'].values.reshape(-1, 1), axis=1)
exog = sm.add_constant(exog)

logit_mod = sm.Logit(endog, exog)
residuals = logit_mod.fit()

residuals.summary()<jupyter_output>Optimization terminated successfully.
         Current function value: 0.369115
         Iterations 7<jupyter_text>Q8. Does neuroticism significantly enhance the model fit when it enters the LR model? To check this, you will need to use the model builder menu to add another block to the model, and re-organise the predictors to address this question.<jupyter_code>endog = df['QIDS_screen'].values

exog = df[questionnaires + ['Age']].values
exog = (exog - exog.mean()) / exog.std()
exog = np.append(exog, df['Gender'].values.reshape(-1, 1), axis=1)
exog = sm.add_constant(exog)

full_model = sm.Logit(endog, exog)
full_residuals = full_model.fit()

constrained_model = sm.Logit(endog, np.delete(exog, 5, 1))
constrained_residuals = constrained_model.fit()
p = ll_ratio_test(
    constrained_residuals.llf,
    full_residuals.llf
)
print(f'Neuroticism: {p}')<jupyter_output>Neuroticism: 1.2828962163015345e-123<jupyter_text>Q9. Use the options in the prediction menu to examine how well this set of predictors classifies cases, relative to the observed data.<jupyter_code>predictions = lgreg().fit(exog, endog).predict_proba(exog)
print(f'f1 score: {f1_score(predictions.argmax(1).reshape(-1, 1), endog)}')<jupyter_output>f1 score: 0.30725190839694655<jupyter_text>Q10: What happens when you change the cutoff value for the classifications to 0.2?<jupyter_code>plt.figure(figsize=(5, 5))
for threshold in [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]:
    
    predictions_new = [1 if i[1] >= threshold else 0 for i in predictions]
    score = f1_score(predictions_new, endog)
    
    plt.scatter(threshold, score, c='r')

plt.xlabel('Threshold')
plt.ylabel('F1 Score')
plt.show()<jupyter_output><empty_output><jupyter_text>Q11: As a final analysis, the team wanted to see if gender moderated the effect of neuroticism on the QIDS screening outcome. Use the model builder to run this analysis in Jamovi. What would you conclude?<jupyter_code>full_model = sm.Logit(endog, exog[:, [0, 5, 7]])
full_residuals = full_model.fit()
    
constrained_mod = sm.Logit(endog, exog[:, [0, 5]])
constrained_residuals = constrained_mod.fit()
constrained_residuals.summary()
full_residuals.summary()
p = ll_ratio_test(
    constrained_residuals.llf,
    full_residuals.llf,
    
)
print('')
print(f'Mediation Neuroticism: {round(p, 3)}')
print('')<jupyter_output>Mediation Neuroticism: 0.761
{'fraction_non_alphanumeric': 0.09055476159542679, 'fraction_numerical': 0.027283357152137196, 'mean_word_length': 4.885321100917431, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '447574', 'license': 'MIT', 'n_tokens_mistral': 2558, 'n_tokens_neox': 2347, 'n_words': 812, 'path': 'advanced_quantitative_methods/units/4_logistic_regression/unit_4_lab_assignement.ipynb', 'repo_name': 'vb690/modules_comp_neuro_2021', 'size': '7697'}