<jupyter_start><jupyter_text>To use t test, first check mean and std on both datasets. Of they on't match, an other statistical test must be used. Other possible test: McNemar.<jupyter_code>for q in shortLikertQuestions:
    d1 = _inputDataSUSNormalized1.dropna().loc[:, q]
    d2 = _inputDataSUSNormalized2.dropna().loc[:, q]
    t2, p2 = stats.ttest_ind(d1,d2)
    print(q)
    print("    t = " + str(t2))
    print("    p = " + str(p2))
    print()
    print()

_inputDataSUSNormalized1.dropna()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1268939393939394, 'fraction_numerical': 0.020833333333333332, 'mean_word_length': 3.990566037735849, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '331973', 'license': 'MIT', 'n_tokens_mistral': 202, 'n_tokens_neox': 200, 'n_words': 49, 'path': 'Functions/6.0 Comparison between datasets.ipynb', 'repo_name': 'raphik12/AMRGamesAnalysis', 'size': '528'}