<jupyter_start><jupyter_text>This notebook shows new functionality of `nideconv` where you can subtract the event-related time courses two conditions from each other and get a corresponding t-value for every timepoint<jupyter_code>import nideconv
from nideconv import simulate
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Here we simulate some data that resembles the design of Failing et al. (2021)<jupyter_code>conditions = [{'name':'plus_type1_exemplar1',
  'mu_group':1,
  'std_group':0.1},
  {'name':'plus_type1_exemplar2',
  'mu_group':1.1,
  'std_group':0.1},
  {'name':'plus_type2_exemplar1',
  'mu_group':1.2,
  'std_group':0.1},
  {'name':'plus_type2_exemplar2',
  'mu_group':1.3,
  'std_group':0.1},
  {'name':'drop_type1_exemplar1',
  'mu_group':.5,
  'std_group':0.1},
  {'name':'drop_type1_exemplar2',
  'mu_group':.5,
  'std_group':0.1},
  {'name':'drop_type2_exemplar1',
  'mu_group':.5,
  'std_group':0.1},
  {'name':'drop_type2_exemplar2',
  'mu_group':.5,
  'std_group':0.1}]
data, onsets, parameters = simulate.simulate_fmri_experiment(conditions, n_rois=2, run_duration=300*7, noise_level=.1)
data.columns = ['voxel 1', 'voxel 2']<jupyter_output><empty_output><jupyter_text>Now, we use `ResponseFitter` to fit a Fourier basis set to all 8 conditions<jupyter_code>fitter = nideconv.ResponseFitter(data, 1.0, )

for condition in conditions:
    fitter.add_event(condition['name'], onsets.loc[condition['name'], 'onset'],
                    interval=[0, 20], basis_set='fourier', n_regressors=6)
    
fitter.fit()<jupyter_output>/Users/gdehol/git/nideconv/nideconv/regressors.py:298: UserWarning: Number of fourier regressors has to be uneven and has automatically been set to 7 per covariate
  warnings.warn('Number of {} regressors has to be uneven and has automatically '<jupyter_text>Subtraction approachThe simplest thing we can do is just subtract the timecourses of the two conditions...First get a DataFrame with all the timecourses...<jupyter_code>tc = fitter.get_timecourses(oversample=5)
tc.head()<jupyter_output><empty_output><jupyter_text>... and then subtract the two conditions from each other. We see that the result has changed...<jupyter_code>tc_plus_type1_exemplar1_corrected = tc.loc['plus_type1_exemplar1'] - tc.loc['drop_type1_exemplar1']<jupyter_output><empty_output><jupyter_text>Indeed, we see that the resulting HRF is lower, because the "visual stimulation"-part of the activation is removed.<jupyter_code>col1 = plt.subplot(121)
tc.loc['plus_type1_exemplar1'].plot(ax=col1)

col2 = plt.subplot(122)
tc_plus_type1_exemplar1_corrected.plot(ax=col2)

plt.gcf().set_size_inches(6, 4)<jupyter_output><empty_output><jupyter_text>This function implements a simple t-test, where the timecourse of `condition2` is subtracted from timecourse of `condition1`. Crucially, the variance and covariance of the two estimates is taken into account, using:$$\text{SEM} = \sqrt{c (X^TX)^{-1}\sigma^2}$$<jupyter_code>t = fitter.t_test('plus_type1_exemplar1', 'drop_type1_exemplar1')
t<jupyter_output><empty_output><jupyter_text>For this very high quality data, the resulting t-values are very high (30).<jupyter_code>t.plot()<jupyter_output><empty_output><jupyter_text>If we simulate noisier data with the same underlying parameters...<jupyter_code>data, onsets, parameters = simulate.simulate_fmri_experiment(conditions, n_rois=2, run_duration=300*7, noise_level=.65)
data.columns = ['voxel 1', 'voxel 2']

fitter = nideconv.ResponseFitter(data, 1.0, )
for condition in conditions:
    fitter.add_event(condition['name'], onsets.loc[condition['name'], 'onset'],
                    interval=[0, 20], basis_set='fourier', n_regressors=6)
    
fitter.fit()

tc = fitter.get_timecourses(oversample=5)

tc_plus_type1_exemplar1_corrected = tc.loc['plus_type1_exemplar1'] - tc.loc['drop_type1_exemplar1'] 

col1 = plt.subplot(121)
tc.loc['plus_type1_exemplar1'].plot(ax=col1)

col2 = plt.subplot(122)
tc_plus_type1_exemplar1_corrected.plot(ax=col2)

plt.gcf().set_size_inches(6, 4)<jupyter_output>/Users/gdehol/git/nideconv/nideconv/regressors.py:298: UserWarning: Number of fourier regressors has to be uneven and has automatically been set to 7 per covariate
  warnings.warn('Number of {} regressors has to be uneven and has automatically '<jupyter_text>the differenc eof the percent signal change stays similar (1. vs .6; although the curves become noisier), but the t-values are much lower. This is because the increased standard errors.<jupyter_code>t = fitter.t_test('plus_type1_exemplar1', 'drop_type1_exemplar1')
t.plot()<jupyter_output>/Users/gdehol/git/nideconv/nideconv/response_fitter.py:238: FutureWarning: Support for multi-dimensional indexing (e.g. `obj[:, None]`) is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.
  (c.values.dot(X_) * c).sum(1).values[:, np.newaxis] * self.sigma2[np.newaxis, :])<jupyter_text>Multiple subjects:<jupyter_code>data, onsets, parameters = simulate.simulate_fmri_experiment(conditions, 
                                                             n_rois=2, 
                                                             run_duration=300*7, 
                                                             noise_level=.1,
                                                             n_subjects=5,
                                                             n_runs=3)
data.columns = ['voxel 1', 'voxel 2']
group_fitter = nideconv.GroupResponseFitter(data, onsets, 1., concatenate_runs=False)
group_fitter.add_event(basis_set='fourier', n_regressors=7, interval=[0, 20])
group_fitter.fit()
t = group_fitter.t_test('plus_type1_exemplar1', 'drop_type1_exemplar1', oversample=3)
import seaborn as sns
sns.lineplot('time', 'voxel 1', 'subject', data=t, palette='Set1')<jupyter_output>/Users/gdehol/miniconda3/lib/python3.8/site-packages/seaborn/_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y, hue. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(
{'fraction_non_alphanumeric': 0.11880064829821718, 'fraction_numerical': 0.029497568881685577, 'mean_word_length': 4.692804428044281, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '371452', 'license': 'MIT', 'n_tokens_mistral': 2216, 'n_tokens_neox': 2106, 'n_words': 471, 'path': 'notebooks/Paired t-test using nideconv.ipynb', 'repo_name': 'VU-Cog-Sci/response_fitter', 'size': '6170'}