<jupyter_start><jupyter_text>Class project grade analysis While grading a class project, I got the feeling that students who chose to use a comic format were having a much harder time demonstrating knowledge and meeting the grading criteria than students who chose to use an essay format. I decided to do a permutation test to determine whether or not the distribution of scores were actually distinct so that I could make an informed decision about whether I needed to adjust my grading.<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
%matplotlib inline<jupyter_output><empty_output><jupyter_text>I recorded each student's score along with the format they chose and their class (two periods did this project)<jupyter_code>df = pd.read_csv("Projects.csv")
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 48 entries, 0 to 47
Data columns (total 4 columns):
Student    48 non-null int64
Class      48 non-null object
Grade      47 non-null float64
Format     47 non-null object
dtypes: float64(1), int64(1), object(2)
memory usage: 1.6+ KB<jupyter_text>One student had not yet submitted the assignment, so I needed to drop their information.<jupyter_code>df.dropna(inplace=True)
df.groupby("Format").count()<jupyter_output><empty_output><jupyter_text>Two students did unique formats, so I dropped their information as well.<jupyter_code>df = df.drop(df[df.Format=="poem"].index)
df = df.drop(df[df.Format=="story"].index)<jupyter_output><empty_output><jupyter_text>The permutation test combines data from two sets of students and splits it back into two random groupings. It is a way to analyze the likelihood that two sets of data really come from the same distribution. My permuttest function performs this analysis, prints the p-value for getting a difference in means as extreme as was seen, and plots a histogram of the difference of means seen during the simulation (which are also returned).<jupyter_code>def permuttest(series1, series2, size=10000):
    """Does a permutation test to compute difference in means between size iterations of a permutation test. Returns array of dif. of means"""
    true_diff = np.mean(series1)-np.mean(series2)
    concatarray = np.concatenate((np.array(series1), np.array(series2)))
    diff_means = np.empty(size)
    for i in range(size):
        permuted = np.random.permutation(concatarray)
        s1 = permuted[:len(series1)]
        s2 = permuted[len(series1):]
        diff_means[i]=np.mean(s1)-np.mean(s2)
    p = sum(np.abs(diff_means)>=true_diff)/size
    print('p-value: {}'.format(p))
    plt.hist(diff_means)
    plt.xlabel("Difference of means")
    plt.ylabel("Frequency")
    plt.axvline(x=true_diff, color="r");
    return diff_means<jupyter_output><empty_output><jupyter_text>The two class periods did not seem to be very different.<jupyter_code>sns.violinplot(x="Class", y="Grade", data=df)
plt.show()
df["Grade"].groupby(df["Class"]).mean()
a=df[df.Class=="A"]["Grade"]
b=df[df.Class=="B"]["Grade"]
diff_means = permuttest(a,b)<jupyter_output>p-value: 1.0<jupyter_text>When I grouped by project format, it seemed like there might be some differences between the formats. The majority of students did either a comic or an essay, and it seemed like the essay writers generally scored better than the comic writers.Although the number of data points in each of these graphs is small and might be better suited for a swarm plot rather than a violin plot, I am keeping it as a violin plot to better maintain the privacy of the students' scores.<jupyter_code>sns.violinplot(x="Format", y="Grade", data=df)
plt.show()
print("Mean grades for each format:")
print(df["Grade"].groupby(df["Format"]).mean())
print("\nMedian grades for each format:")
print(df["Grade"].groupby(df["Format"]).median())<jupyter_output><empty_output><jupyter_text>The average comic scored 8 points less than the average essay.<jupyter_code>e=df[df.Format=="essay"]["Grade"]
c=df[df.Format=="comic"]["Grade"]
essay_comic_dif = np.mean(e)-np.mean(c)
print(essay_comic_dif)
diff_means = permuttest(e,c)
np.percentile(diff_means, [2.5, 97.5])<jupyter_output><empty_output><jupyter_text>Although it appeared that there might be a difference in the students' scores based on their format choice, it was not significant. I also tested the difference in the median scores, as there were a few outliers who brought down the mean scores. However, it was also not significant.<jupyter_code>def permuttestmedian(series1, series2, size=10000):
    """Does a permutation test to compute difference in medians between size iterations of a permutation test. Returns array of dif. of medians"""
    true_diff = np.median(series1)-np.median(series2)
    concatarray = np.concatenate((np.array(series1), np.array(series2)))
    diff_medians = np.empty(size)
    for i in range(size):
        permuted = np.random.permutation(concatarray)
        s1 = permuted[:len(series1)]
        s2 = permuted[len(series1):]
        diff_means[i]=np.median(s1)-np.median(s2)
    p = sum(np.abs(diff_means)>=true_diff)/size
    print('p-value: {}'.format(p))
    plt.hist(diff_means)
    plt.xlabel("Difference of medians")
    plt.ylabel("Frequency")
    plt.axvline(x=true_diff, color="r");
    return diff_medians
diff_medians = permuttestmedian(e,c)<jupyter_output>p-value: 0.1099
{'fraction_non_alphanumeric': 0.10661696178937559, 'fraction_numerical': 0.013793103448275862, 'mean_word_length': 5.2540792540792545, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '519247', 'license': 'MIT', 'n_tokens_mistral': 1748, 'n_tokens_neox': 1646, 'n_words': 554, 'path': 'Class_projects.ipynb', 'repo_name': 'hjohnsen/classroom_analysis', 'size': '5365'}