<jupyter_start><jupyter_text>Standard Deviation of Gross Conversion:<jupyter_code>round(np.sqrt((.206250*(1-.206250))/(5000*3200/40000)),4)<jupyter_output><empty_output><jupyter_text>Standard Deviation of Retention:<jupyter_code>round(np.sqrt((0.530000*(1-0.530000))/(5000*660/40000)),4)<jupyter_output><empty_output><jupyter_text>Standard Deviation of Net Conversion:<jupyter_code>round(np.sqrt((0.109313*(1-0.109313))/(5000*3200/40000)),4)<jupyter_output><empty_output><jupyter_text>http://www.evanmiller.org/ab-testing/sample-size.html  Pageviews without Bonferroni Correction Gross ConversionBaseline Conversion Rate: 20.625%Minimum Detectable Effect: 1%α: 5%β: 20%1 - β: 80%Sample Size = 25,835 (Enrollments per Group)Number of Groups = 2 (Experiment and Control)Total Sample Size = 51,670 enrollmentsClicks/Pageview: 3200/40000 = 0.08 Clicks/PageviewPageviews = 645,875 RetentionBaseline Conversion Rate: 53%Minimum Detectable Effect: 1%α: 5%β: 20%1 - β: 80%Sample Size = 39,115 (Enrollments per Group)Number of Groups = 2 (Experiment and Control)Total Sample Size = 78,230 enrollmentsClicks/Pageview: 660/40000 =  0.0165 Clicks/PageviewPageviews = 4,741,212 Net ConversionBaseline Conversion Rate: 10.9313%Minimum Detectable Effect: 0.75%α: 5%β: 20%1 - β: 80%Sample Size = 27,413 (Enrollments per Group)Number of Groups = 2 (Experiment and Control)Total Sample Size = 54,826 enrollmentsClicks/Pageview: 3200/40000 = 0.08 Clicks/PageviewPageviews = 685,325  Duration and Exposure Number of Pageview: 685,325 Fraction of Traffic Exposed:<jupyter_code>685325.0/40000<jupyter_output><empty_output><jupyter_text>Length of Experiment: 18 days<jupyter_code>df_control = pd.read_csv("Final Project Results - Control.csv")
df_experiment = pd.read_csv("Final Project Results - Experiment.csv")
results = {"Control":pd.Series([df_control.Pageviews.sum(),df_control.Clicks.sum(),
                                  df_control.Enrollments.sum(),df_control.Payments.sum()],
                                  index = ["Cookies","Clicks","Enrollments","Payments"]),
           "Experiment":pd.Series([df_experiment.Pageviews.sum(),df_experiment.Clicks.sum(),
                               df_experiment.Enrollments.sum(),df_experiment.Payments.sum()],
                               index = ["Cookies","Clicks","Enrollments","Payments"])}
df_results = pd.DataFrame(results)
df_results<jupyter_output><empty_output><jupyter_text>Sanity Checks<jupyter_code>df_results['Total'] = df_results.Control + df_results.Experiment
df_results['Prob'] = 0.5
df_results['StdErr'] = np.sqrt((df_results.Prob * (1-df_results.Prob))/df_results.Total)
df_results['MargErr'] = 1.96 * df_results.StdErr
df_results['CI_lower'] = df_results.Prob - df_results.MargErr
df_results['CI_upper'] = df_results.Prob + df_results.MargErr
df_results['Obs_val'] = df_results.Experiment/df_results.Total
df_results['Pass_Sanity'] = df_results.apply(lambda x: (x.Obs_val > x.CI_lower) and (x.Obs_val < x.CI_upper), axis=1)
df_results['Diff'] = abs((df_results.Experiment - df_results.Control)/df_results.Total)

df_results<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13545688545688545, 'fraction_numerical': 0.07593307593307594, 'mean_word_length': 5.686021505376344, 'pattern_counts': {'":': 2, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '409580', 'license': 'MIT', 'n_tokens_mistral': 1265, 'n_tokens_neox': 1063, 'n_words': 176, 'path': 'p7_design_an_ab_test/design_an_ab_test.ipynb', 'repo_name': 'TokyoIndex/dand', 'size': '3108'}