<jupyter_start><jupyter_text>How to inspect model fit results<jupyter_code>import rlssm

# load non-hierarchical DDM fit:
model_fit_ddm = rlssm.load_model_results('/Users/laurafontanesi/git/rlssm/docs/notebooks/DDM.pkl')

# load non-hierarchical LBA fit:
model_fit_lba = rlssm.load_model_results('/Users/laurafontanesi/git/rlssm/docs/notebooks/LBA_2A.pkl')

# load hierarchical RL fit:
model_fit_rl = rlssm.load_model_results('/Users/laurafontanesi/git/rlssm/docs/notebooks/hierRL_2A.pkl')<jupyter_output><empty_output><jupyter_text>Posteriors The posterior samples are stored in `samples`:<jupyter_code>model_fit_ddm.samples
model_fit_rl.samples.describe()<jupyter_output><empty_output><jupyter_text>You can simply plot the model's posteriors using `plot_posteriors`:<jupyter_code>model_fit_ddm.plot_posteriors();<jupyter_output><empty_output><jupyter_text>By default, 95% HDIs are shown, but you can also choose to have the posteriors without intervals or BCIs, and change the alpha level:<jupyter_code>model_fit_rl.plot_posteriors(show_intervals='BCI', alpha_intervals=.01);<jupyter_output><empty_output><jupyter_text>Trial-levelDepending on the model specification, you can also extract certain trial-level parameters as numpy ordered dictionaries of n_samples X n_trials shape:<jupyter_code>model_fit_ddm.trial_samples['drift_t'].shape
model_fit_ddm.trial_samples.keys()
model_fit_lba.trial_samples.keys() # for the LBA<jupyter_output><empty_output><jupyter_text>In the case of a RL model fit on choices alone, you can extract the log probability of accuracy=1 for each trial:<jupyter_code>model_fit_rl.trial_samples.keys()
model_fit_rl.trial_samples['log_p_t'].shape<jupyter_output><empty_output><jupyter_text>Posterior predictives With `get_posterior_predictives_df` you get posterior predictives as pandas DataFrames of `n_posterior_predictives` X `n_trials` shape:<jupyter_code>pp = model_fit_rl.get_posterior_predictives_df(n_posterior_predictives=1000)
pp<jupyter_output><empty_output><jupyter_text>For the DDM, you have additional parameters to tweak the DDM simulations, and you get a DataFrame with a hierarchical column index,  for RTs and for accuracy:<jupyter_code>pp = model_fit_ddm.get_posterior_predictives_df(n_posterior_predictives=100, dt=.001)
pp<jupyter_output><empty_output><jupyter_text>You can also have posterior predictive summaries with `get_posterior_predictives_summary`.Only mean accuracy for RL models fit on choices alone, and also mean RTs, skewness and quantiles for lower and upper boundaries for models fitted on RTs as well.<jupyter_code>model_fit_rl.get_posterior_predictives_summary()
model_fit_ddm.get_posterior_predictives_summary()<jupyter_output><empty_output><jupyter_text>You can also specify which quantiles you are interested in:<jupyter_code>model_fit_lba.get_posterior_predictives_summary(n_posterior_predictives=200, quantiles=[.1, .5, .9])<jupyter_output><empty_output><jupyter_text>Finally, you can get summary for grouping variables (e.g., experimental conditions, trial blocks, etc.) in your data:<jupyter_code>model_fit_lba.get_grouped_posterior_predictives_summary(n_posterior_predictives=200,
                                                        grouping_vars=['block_label'],
                                                        quantiles=[.3, .5, .7])<jupyter_output><empty_output><jupyter_text>Plot posterior predictives You can plot posterior predictives similarly, both **ungrouped** (across all trials) or **grouped** (across conditions, trial blocks, etc.plot_mean_posterior_predictives).For RT models, you have both **mean plots**, and **quantile plots**:<jupyter_code>model_fit_ddm.plot_mean_posterior_predictives(n_posterior_predictives=200);<jupyter_output><empty_output><jupyter_text>Quantile plots have 2 main visualization options, "shades" and "lines", and you can specify again which quantiles you want, which in tervals and alpha levels:<jupyter_code>model_fit_lba.plot_quantiles_posterior_predictives(n_posterior_predictives=200);
model_fit_lba.plot_quantiles_posterior_predictives(n_posterior_predictives=200,
                                                   kind='shades',
                                                   quantiles=[.1, .5, .9]);
model_fit_lba.plot_quantiles_grouped_posterior_predictives(
    n_posterior_predictives=100,
    grouping_var='block_label',
    kind='shades',
    quantiles=[.1, .3, .5, .7, .9]);
# Define new grouping variables:

import pandas as pd
import numpy as np

data = model_fit_rl.data_info['data']

# add a column to the data to group trials across learning blocks
data['block_bins'] = pd.cut(data.trial_block, 8, labels=np.arange(1, 9))

# add a column to define which choice pair is shown in that trial
data['choice_pair'] = 'AB'
data.loc[(data.cor_option == 3) & (data.inc_option == 1), 'choice_pair'] = 'AC'
data.loc[(data.cor_option == 4) & (data.inc_option == 2), 'choice_pair'] = 'BD'
data.loc[(data.cor_option == 4) & (data.inc_option == 3), 'choice_pair'] = 'CD'
import matplotlib.pyplot as plt
import seaborn as sns

fig, axes = plt.subplots(1, 2, figsize=(20,8))

model_fit_rl.plot_mean_grouped_posterior_predictives(grouping_vars=['block_bins'], n_posterior_predictives=500, ax=axes[0])

model_fit_rl.plot_mean_grouped_posterior_predictives(grouping_vars=['block_bins', 'choice_pair'], 
                                                     n_posterior_predictives=500, ax=axes[1])

sns.despine()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0963436928702011, 'fraction_numerical': 0.01316270566727605, 'mean_word_length': 5.830212234706616, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '527708', 'license': 'MIT', 'n_tokens_mistral': 1915, 'n_tokens_neox': 1808, 'n_words': 380, 'path': 'docs/notebooks/inspect_model.ipynb', 'repo_name': 'birgovanandrei/rlssm', 'size': '5470'}