<jupyter_start><jupyter_text>RANTEST performs a randomisation test to compare two independent samples.  According to the null hypothesis of    no-difference, each outcome would have been the same regardless of which group the individual happened to be allocated to.  Therefore all N = n1 + n2 observations are pooled and, as in the actual experiment, divided at random into groups of size n1 and n2.  The fraction of randomisations that gives rise to a difference between the groups at least as large as that observed gives the P value.In the binomial case, in which the measurement is the fraction of 'successes' in each sample (say r1 out of n1, and r2 out of n2) a 'success' is given a score of 1, 'failure' scores 0.Randomisation test on binomial data could be done using as criterion: (1) number of successes in set 1 (r1) or (2) difference between the p=r/n values.Both criteria give the same one-tail P value. Use of r1 as criterion is the direct Monte Carlo equivalent of summing the the exact Fisher test probabilities for the observed 2x2 table with those for all tables that depart further from the null hypothesis in the observed direction. A 2-tail probablilty can be found by doubling the one-tail value, at least if the displayed distribution is symmetrical. Use of (p1-p2) as criterion gives both one and two-tail     probabilities directly by seeing how many random allocations of the observations to groups of size n1 and n2 produce and absolute value of (p1-p2) at least as big as that observed.<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import sys
import math
import random
import numpy as np
#from dcstats.rantest import RantestBinomial
from dcstats.basic_stats import TTestBinomial
class RantestBinomial():
    def __init__(self, ir1, if1, ir2, if2):
        """ 
        Parameters
        ----------
        ir1 : number of successes in first trial, int
        if1 : number of failures in first trial, int
        ir2 : number of successes in second trial, int
        if2 : number of failures in second trial, int       
        """
        self.ir1, self.if1 = ir1, if1
        self.ir2, self.if2 = ir2, if2
        self.ir = ir1 + ir2
        self.n1 = ir1 + if1 # tot number of tests in first trial 
        self.n2 = ir2 + if2 # tot number of tests in second trial
        self.ntot = self.n1 + self.n2
        self.dobs = ir1 / float(self.n1) - ir2 / float(self.n2)
        random.seed(1984)
        
    def run_rantest(self, nran):
        self.nran = nran
        self.randiff = np.zeros(nran) # difference between means
        allobs = [1]*self.ir + [0]*(self.ntot - self.ir)
        for k in range(0, self.nran):
            random.shuffle(allobs)
            is2 = sum(allobs[self.n1:])
            self.randiff[k] = (self.ir - is2) / float(self.n1) - is2 / float(self.n2)
        self.ng1 = self.randiff[self.randiff >= self.dobs].size
        self.ne1 = self.randiff[self.randiff == self.dobs].size
        self.nl1 = self.randiff[self.randiff <= self.dobs].size
        
    def __repr__(self):        
        return ('\n\n Rantest:  {0:d} randomisations:'.format(self.nran) +
            '\n P values for difference between sets are:' +
            '\n  r1 greater than or equal to observed: P = {0:.6f}'.format(self.ng1 / float(self.nran)) +
            '\n  r1 less than or equal to observed: P = {0:.6f}'.format(self.nl1 / float(self.nran)) +
            '\n  r1 equal to observed: number = {0:d} (P = {1:.6f})'.format(self.ne1, self.ne1 / float(self.nran)))

ir1, if1 = 3, 4 # Number of successes and failures in trial 1
ir2, if2 = 4, 5 # Number of successes and failures in trial 2<jupyter_output><empty_output><jupyter_text>Calculate basic statistics<jupyter_code>ttb = TTestBinomial(ir1, if1, ir2, if2)
print(ttb)<jupyter_output>Set 1: 3 successes out of 7;
 p1 = 0.428571;   SD(p1) = 0.187044
 Set 2: 4 successes out of 9;
 p2 = 0.444444;   SD(p2) = 0.165635
 Observed difference between sets, p1-p2 = -0.015873

 Observed 2x2 table:
  Set 1:    3      4      7
  Set 2:    4      5      9
  Total:    7      9      16

 Two-sample unpaired test using Gaussian approximation to binomial:
 standard normal deviate = 0.063492; two tail P = 0.949375.<jupyter_text>Run randomisation tests<jupyter_code>nran = 50000
rnt = RantestBinomial(ir1, if1, ir2, if2)
rnt.run_rantest(nran)
print(rnt)<jupyter_output>Rantest:  50000 randomisations:
 P values for difference between sets are:
  r1 greater than or equal to observed: P = 0.713320
  r1 less than or equal to observed: P = 0.672160
  r1 equal to observed: number = 19274 (P = 0.385480)
{'fraction_non_alphanumeric': 0.08664495114006515, 'fraction_numerical': 0.047122692725298586, 'mean_word_length': 2.930034129692833, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '643414', 'license': 'MIT', 'n_tokens_mistral': 1567, 'n_tokens_neox': 1446, 'n_words': 576, 'path': 'example- Randomisation test of binomial data.ipynb', 'repo_name': 'remislp/DC-Stats-for-Mac', 'size': '4605'}