<jupyter_start><jupyter_text>Conditional Stop Probability> Probability of stopping recall conditional on number of items recalled so far  Data PreparationFor our demonstrations, we'll lean on the MurdockOkada1970 dataset. As a reminder, in this dataset each of 72 undergraduates was given 20 test lists with 20-word lists visually presented at either 60 or 120 words/min.<jupyter_code>from compmemlearn.datasets import prepare_murdock1970_data

trials, events, list_length = prepare_murdock1970_data('../../data/mo1970.txt')
events.head()<jupyter_output><empty_output><jupyter_text>Analysis  Fast Array Generation`DataFrame`s contain granular subject-by-subject information and are easier to plot using the `seaborn` library. But sometimes we don't need this granular information and mainly want to perform our analysis as quickly as possible -- perhaps to help with model fitting or analysis. In that case, representing results with `numpy` arrays and performing just-in-time compilation of our function using `numba` might be preferred. We include `analyses.fast_csp` in our library for that purpose here.<jupyter_code># export

from numba import njit
import numpy as np

@njit(fastmath=True, nogil=True)
def fast_csp(trials, item_count):
    
    # numerator is number of trials with zero in current column position
    numerator = np.zeros(item_count+1)

    # denominator is number of trials without zero in previous column positions
    denominator = np.zeros(item_count+1)

    stop_positions = trials == 0
    for i in range(len(trials)):

        # add 1 to index of final recall
        numerator[np.argmax(stop_positions[i])] += 1
        
        # add 1 to each index up through final recall
        denominator[:np.argmax(stop_positions[i])+1] += 1
        
    denominator[denominator==0] += 1
    return numerator/denominator
fast_csp(trials, list_length)<jupyter_output><empty_output><jupyter_text>DataFrameThe `psifr` library doesn't have a function to generate `pandas` DataFrames containing conditional stop probability information, so we make our own. For efficiency, it mainly consists of calls to `fast_csp`.<jupyter_code>import pandas as pd

def csp(events, trials):
    subjects = len(np.unique(events.subject))
    trial_count = np.max(events.list)
    list_length = np.max(events.input)
    
    result = {'subject': [], 'output': [], 'prob': []}
    
    for subject in range(subjects):
        subject_result = fast_csp(trials[subject*trial_count:(subject+1)*trial_count], list_length)
        subject_result[np.argmax(subject_result)+1:] = np.nan

        result['subject'] += [subject+1]*(list_length+1)
        result['output'] += np.arange(list_length+1, dtype=int).tolist()
        result['prob'] += subject_result.tolist()
        
    return pd.DataFrame(result)
csp(events, trials)<jupyter_output><empty_output><jupyter_text>We can compare the runtimes of our implementations using the `%%timeit` Jupyter magic:<jupyter_code>%%timeit
fast_csp(trials, list_length)
%%timeit
csp(events, trials)<jupyter_output>2.53 ms ± 156 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)<jupyter_text>`fast_csp` is slower than our serial position effect functions, but faster than our functions analyzing lag-contiguity.  PlottingWe'll define a `seaborn`-based plotting function that uses our `csp` function.<jupyter_code># export
import seaborn as sns
from psifr import fr

def plot_csp(data, **facet_kws):
    
    trials = pd.pivot_table(
        data, index=['subject', 'list'], 
        columns=['output'], values='input', 
        fill_value=0).to_numpy()

    sns.lineplot(
        data=csp(data, trials), 
        x='output', y='prob', **facet_kws)
sns.set_theme(style="darkgrid")

g = sns.FacetGrid(dropna=False, data=events, height=4, aspect=1.2)
g.map_dataframe(plot_csp, err_style='bars')

g.set_xlabels('Recall Position')
g.set_ylabels('Conditional Stop Probability');<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10053313023610053, 'fraction_numerical': 0.014216806296014217, 'mean_word_length': 4.660919540229885, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '166504', 'license': 'MIT', 'n_tokens_mistral': 1281, 'n_tokens_neox': 1205, 'n_words': 368, 'path': 'library/analyses/Conditional_Stop_Probability.ipynb', 'repo_name': 'vucml/compmemlearn', 'size': '3939'}