<jupyter_start><jupyter_text>Second supplement, data analysisEarlier topic models didn't include all the volumes we would need to test our hypotheses. The model produced with a "second supplement" of volumes is the first one that covers all the bases. This is an initial analysis of information-theoretic asymmetries in that model.For the code used to measure the asymmetries, see ```../entropycalc.``` This analysis begins after that code has run, producing a series of "summary files" that describe novelty, transience, and (what Barron et al call) "resonance" for each volume. We are probably going to rename resonance something like "anticipation."First, we import a few models that will prove useful later.<jupyter_code>import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
import glob, math, random
from scipy.stats import pearsonr, zscore, ttest_ind
from statistics import mean, stdev
from math import sqrt<jupyter_output><empty_output><jupyter_text>get metadataIn order to analyze the data we will need information about date of publication for each volume (technically "latest possible date of composition," which is defined as the *earlier* of date of publication or author's date of death if known).We're also going to need information about / predictions of nationality, which are collated in [```detect_americans```](https://github.com/tedunderwood/asymmetry/blob/master/analysis/detect_americans.ipynb).<jupyter_code>meta = pd.read_csv('../supplement2/supp2nationalitymeta.tsv', sep = '\t', index_col = 'docid')<jupyter_output><empty_output><jupyter_text>get dataNow the data itself. This is broken into a number of "summary files" because the processing that produced it had to be distributed across a cluster. (The entropy calculation is done by comparing individual volumes to each other, and when you've got 40k vols, the number of cross-comparisons becomes fairly large.)So we first make a list of all the files we need ...<jupyter_code>paths = glob.glob('../supp2results/*summary.tsv')
paths<jupyter_output><empty_output><jupyter_text>... and then loop across the list, reading them in ... and finally concatenate the data frames.<jupyter_code>dfs = []
for p in paths:
    df = pd.read_csv(p, sep = '\t', index_col = 'docid')
    dfs.append(df)
    print(df.shape)

data = pd.concat(dfs, verify_integrity = True)
print(data.shape)<jupyter_output>(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(1817, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(2000, 36)
(39817, 36)<jupyter_text>Map a couple of columns from the metadata into our dataWe're going to use **latest-possible-date-of-composition** for our analysis.<jupyter_code>data = data.join(meta.latestcomp, how = 'inner')<jupyter_output><empty_output><jupyter_text>We're also going to divide **US authors** from others, because    1. A lot of our hypotheses are US-specific, and    2. resonance/anticipation tends to be slightly higher for US authors than for others, simply because the US fraction of the library **increases** across time.Given those two facts, it gives us slightly more confidence to calculate resonance/anticipation separately for US and for non-US authors. We could in principle also separate authors who are British, Canadian, Indian, and so forth, but since we have few hypotheses specific to those nationalities, the gains are likely to be outweighed by increases in fragility.The nationality column of the metadata has a lot of possibilities; to simplify the next step, we're going to reduce it to a binary.<jupyter_code>def whether_us(anation):
    if anation.startswith('us') or anation == 'guess: us':
        return True
    else:
        return False

meta = meta.assign(isus = meta.nationality.map(whether_us))
sum(meta.isus)<jupyter_output><empty_output><jupyter_text>In short, we've got about 18,000 volumes by US authors, in about 40,000 vols total. Note that a lot of these nationalities are inferred / estimated. Overall accuracy (combining manual ground truth and estimation) comes to about 88-89%; consult [```detect_americans```](https://github.com/tedunderwood/asymmetry/blob/master/analysis/detect_americans.ipynb) for details.We are also fully aware that people like <NAME> move across the Atlantic halfway through their lives, that some volumes are collections combining authors of different nationalities, and so on. If we were to fully describe the nuances of each volume, a single "nationality" code would be grossly inadequate. To provide that full description across 40,000 volumes would also take several million pages, and you would quickly stop reading!Nationality is not a central subject of inquiry in this project. We're generating a simplified, imperfect *model* of nationality merely because it helps us (imperfectly) factor out a confounding variable that has a modest effect on our results. We think that provides a slight interpretive advantage, but the conclusions of our study would not be profoundly altered if we ignored nationality; to estimate the size of the effect, you can consult [earlier notebooks and models where nationality is not factored out.](https://github.com/tedunderwood/asymmetry/blob/master/analysis/first_eda.ipynb)<jupyter_code>data = data.join(meta.isus, how = 'inner')
data.shape<jupyter_output><empty_output><jupyter_text>Normalize resonance for date and nationality"Normalize" here means that we take a 7-year window of volumes that are by US authors (or non-US authors), and calculate z scores for volumes within that window. (I.e., subtract the mean and divide by standard deviation.) We replace the raw resonance/anticipation score for each volume with the z score calculated when it's at the center of the window.I've already explained why we normalize for nationality. The reason for normalizing by date is that distance calculations on topic vectors cannot be trusted to remain uniform across a timeline. There are edge-sampling effects which make distances lower toward the ends of the timeline. This is true for both cosine distance and KLD.Here's the loop where we actually do the normalizing. This does take some time to run (~20 min). There might be a simpler/quicker way to do this with an ```apply``` method, but it's a bit tricky, since we use a 7-year span to generate the z scores, but only copy over the scores for the central year.The result is a new dataframe called ```zdata.``` This is the data that will be used in subsequent analyses.In doing this, we also loop across columns and calculate a separate z score for each column. The reason is that resonance/anticipation can be calculated in a range of different ways--using different fractions of the dataset, and different temporal windows. There's a separate column for each of these possibilities. (Note, however, that we preregistered some guidelines about the modes of calculation we would use in checking hypotheses, to avoid a garden of infinitely forking paths.)<jupyter_code>zdata = data.copy(deep = True)
columns = [x for x in zdata.columns.tolist() if x.startswith('resonance')]

for col in columns:
    zdata[col] = np.nan
    # set default as empty

for yankeeness in [True, False]:
    for yr in range(1800, 2009):
        if yr % 50 == 1:
            print(yr)
        df = data.loc[(data.latestcomp >= yr - 3) & (data.latestcomp <= yr + 3) & (data.isus == yankeeness), : ]
        for col in columns:
            nas = np.isnan(df[col])
            seriestonormalize = df.loc[~nas, col]
            indices = seriestonormalize.index.values
            zscores = zscore(seriestonormalize)
            for idx, z in zip(indices, zscores):
                date = df.loc[idx, 'latestcomp']
                if date == yr:
                    zdata.loc[idx, col] = z<jupyter_output>1801
1851
1901
1951
2001
1801
1851
1901
1951
2001<jupyter_text>I would guess that warning means I need to make the window a little larger in the early going.  AnalysisOur hypotheses were [preregistered with the Open Science Framework.](https://osf.io/zuq9a/register/5771ca429ad5a1020de2872e) (See "how many and which conditions will participants be assigned to?")We'll test them more or less in order. first hypothesis: reprintingTo start, the hypothesis about reprinting. For this we'll need information about the number of subsequent copies associated with each title. (To see where this is calculated, you'll need to consult the [```noveltmmeta```](https://github.com/tedunderwood/noveltmmeta) repo.)<jupyter_code>zdata = zdata.join(meta.allcopiesofwork, how = 'inner')
zdata.head()<jupyter_output><empty_output><jupyter_text>Now we loop across resonance columns. For each column we remove NaNs, and then calculate correlation with the number of copies associated with volumes.Since reprinting is sort of a feast-or-famine thing, we also try correlation with log(num_vols), which indeed is slightly stronger.<jupyter_code>for col in columns:
    nas = np.isnan(zdata[col]) | np.isnan(zdata['allcopiesofwork'])
    r, p = pearsonr(zdata.loc[~nas, col], zdata.loc[~nas, 'allcopiesofwork'])
    print(col, r, p)
    r, p = pearsonr(zdata.loc[~nas, col], np.log(zdata.loc[~nas, 'allcopiesofwork'] + 0.64))
    print("log-transformed: ", col, r, p)
    print()<jupyter_output>resonance_1.0_10 0.0953931234566 7.77876227474e-75
log-transformed:  resonance_1.0_10 0.107388427978 1.90844556743e-94

resonance_1.0_25 0.0935771913294 1.33042731513e-63
log-transformed:  resonance_1.0_25 0.11447631186 1.86808398768e-94

resonance_1.0_40 0.0844974216679 5.10923172766e-43
log-transformed:  resonance_1.0_40 0.108920089117 1.88275759494e-70

resonance_0.2_10 0.115465304155 5.55746452309e-109
log-transformed:  resonance_0.2_10 0.120951970388 1.80197609569e-119

resonance_0.2_25 0.110813779942 1.36188617721e-88
log-transformed:  resonance_0.2_25 0.12616547101 1.79545832372e-114

resonance_0.2_40 0.1004553444 3.77330412289e-60
log-transformed:  resonance_0.2_40 0.120413581365 8.46942351179e-86

resonance_0.05_10 0.112353549022 2.967808422e-103
log-transformed:  resonance_0.05_10 0.12017833202 5.82065401123e-118

resonance_0.05_25 0.122453978833 6.69996349584e-108
log-transformed:  resonance_0.05_25 0.135938700606 9.58291528736e-133

resonance_0.05_40 0.115321908547 8.325597[...]<jupyter_text>We calculated resonance for 100% of the corpus, as well as the 20%, 5% and 2.5% of volumes closest to the volume being evaluated.We also measured across 10-, 25-, and 40-year windows.**Notice, however, that the differences between different measurement strategies are on the whole rather modest!** This is not necessarily true for individual vols, but when you're looking at broad correlations or differences of means between groups, the measurement strategy is rarely determinative.  second and other hypothesesWill require more metadata about the volumes contained in particular groups (bestsellers, mostdiscussed, etc). We load this below.<jupyter_code>hypothesis_meta = pd.read_csv('../meta/second_supplement_maxoverlap.tsv', sep = '\t')
hypothesis_meta.columns.tolist()<jupyter_output><empty_output><jupyter_text>The last eight columns above describe particular groups to be used in testing hypotheses. Rows (volumes) will have True or False in each of these columns. second hypothesisThe hypothesis about reviewing is a little different than the others, because our test set in this case comes with a contrast set that was constructed manually to match its distribution across dates. So we can use one column to create the test set, another to create the contrast set, and then simply compare the groups.<jupyter_code>reviewed_docs = hypothesis_meta.loc[hypothesis_meta.reviewed == True, 'docid']
unreviewed_docs = hypothesis_meta.loc[hypothesis_meta.contrast4reviewed == True, 'docid']

for col in columns:
    reviewed_data = zdata.loc[reviewed_docs, col]
    reviewed_data = reviewed_data[~np.isnan(reviewed_data)]
    unreviewed_data = zdata.loc[unreviewed_docs, col]
    unreviewed_data = unreviewed_data[~np.isnan(unreviewed_data)]
    t, p = ttest_ind(reviewed_data, unreviewed_data)
    print(col, "t-test", t, p)
    a = reviewed_data
    b = unreviewed_data
    cohens_d = (mean(a) - mean(b)) / (sqrt((stdev(a) ** 2 + stdev(b) ** 2) / 2))
    print(col, "Cohen's d", cohens_d)
    print()<jupyter_output>resonance_1.0_10 t-test 8.18489644079 7.17697465673e-16
resonance_1.0_10 Cohen's d 0.484656100018

resonance_1.0_25 t-test 8.71311330151 1.01469822134e-17
resonance_1.0_25 Cohen's d 0.51603188055

resonance_1.0_40 t-test 8.37203403202 1.62931510126e-16
resonance_1.0_40 Cohen's d 0.495506207256

resonance_0.2_10 t-test 7.46905418059 1.59358708582e-13
resonance_0.2_10 Cohen's d 0.442330617997

resonance_0.2_25 t-test 8.5101111178 5.35650755746e-17
resonance_0.2_25 Cohen's d 0.50411223647

resonance_0.2_40 t-test 8.1844552086 7.20186320816e-16
resonance_0.2_40 Cohen's d 0.484619944984

resonance_0.05_10 t-test 7.4205011709 2.26330319651e-13
resonance_0.05_10 Cohen's d 0.438918048083

resonance_0.05_25 t-test 9.07193344903 4.94154737988e-19
resonance_0.05_25 Cohen's d 0.53724385254

resonance_0.05_40 t-test 8.51814366021 5.01843271806e-17
resonance_0.05_40 Cohen's d 0.504591225364

resonance_0.025_10 t-test 6.96516472852 5.50970318492e-12
resonance_0.025_10 Cohen's d 0.411950304381

resona[...]<jupyter_text>functions used to test subsequent hypothesesThe other hypotheses we preregistered don't come with a predefined contrast set. Instead we specify a control set that matches the test set's distribution across the timeline, and excludes authors included in the test set. Ensuring the latter condition is tricky, but also not likely to make a huge difference given the low odds of random collision in this space. For the moment, let's just say, our contrast set will be selected by matching dates.We can write a function to do this selection.<jupyter_code>def main_and_contrast_set(categorylist, hypmeta, data):
    ''' Accepts a list of categories and finds volumes matching those categories in 
    *hypmeta*. Then constructs a set of matching volumes.
    '''
    
    vols_in_cat = []
    dates_of_cat = []
    for cat in categorylist:
        vols_in_cat.extend(hypmeta.loc[hypmeta[cat] == True, 'docid'])
        dates_of_cat.extend(hypmeta.loc[hypmeta[cat] == True, 'latestcomp'])
    
    # TODO: Right now, since categories overlap, some volumes can be represented more
    # than once. Need to fix this.
        
    contrast_vols = []
    for d in dates_of_cat:
        population = data.index[data['latestcomp'] == d].tolist()
        population = set(population) - set(vols_in_cat)
        if len(population) < 1:
            print('*')
            continue
        else:
            chosen = random.sample(population, 1)[0]
            contrast_vols.append(chosen)
    
    return vols_in_cat, contrast_vols
    
    
def general_test(acat, bcat, zdata): 
    ''' Given two sets of volumes, calculates t tests and
    Cohen's d for difference of means.
    '''
    
    global columns
    for col in columns:
        a = zdata.loc[acat, col]
        a = a[~np.isnan(a)]
        b = zdata.loc[bcat, col]
        b = b[~np.isnan(b)]
        t, p = ttest_ind(a, b)
        print(col, "t-test", t, p)
        cohens_d = (mean(a) - mean(b)) / (sqrt((stdev(a) ** 2 + stdev(b) ** 2) / 2))
        print(col, "Cohen's d", cohens_d)
        print()<jupyter_output><empty_output><jupyter_text>Best sellersThis is not technically one of our preregistered hypotheses, because we weren't confident we knew what to expect here. We mention that we're going to test it, and say that we don't know whether there will be an effect.<jupyter_code>bestsellers, notbestsellers = main_and_contrast_set(['bestseller'], hypothesis_meta, zdata)
general_test(bestsellers, notbestsellers, zdata)<jupyter_output>*
*
*
resonance_1.0_10 t-test 6.43007915411 1.67190978433e-10
resonance_1.0_10 Cohen's d 0.318634771171

resonance_1.0_25 t-test 7.10852024775 1.83227488482e-12
resonance_1.0_25 Cohen's d 0.372587699362

resonance_1.0_40 t-test 6.30153084753 4.03142788474e-10
resonance_1.0_40 Cohen's d 0.34993414797

resonance_0.2_10 t-test 5.91441105179 4.05132712186e-09
resonance_0.2_10 Cohen's d 0.293079008322

resonance_0.2_25 t-test 6.08722988243 1.46713177541e-09
resonance_0.2_25 Cohen's d 0.319057539169

resonance_0.2_40 t-test 5.55720429346 3.3244507281e-08
resonance_0.2_40 Cohen's d 0.308601285324

resonance_0.05_10 t-test 5.49343231996 4.56754238828e-08
resonance_0.05_10 Cohen's d 0.272212597703

resonance_0.05_25 t-test 6.33150578793 3.22773215249e-10
resonance_0.05_25 Cohen's d 0.331861075555

resonance_0.05_40 t-test 5.951514683 3.41543041064e-09
resonance_0.05_40 Cohen's d 0.33050608699

resonance_0.025_10 t-test 4.8823501877 1.1511762453e-06
resonance_0.025_10 Cohen's d 0.241934725263

r[...]<jupyter_text>Third hypothesis: volumes canonized by NortonHere we're adding Heath and Norton Short Fiction; we should separate those in further analysis.<jupyter_code>canon, notcanon = main_and_contrast_set(['norton', 'heath', 'nortonshort'], hypothesis_meta, zdata)
general_test(canon, notcanon, zdata)<jupyter_output>resonance_1.0_10 t-test 1.18215752533 0.238499377839
resonance_1.0_10 Cohen's d 0.163574074763

resonance_1.0_25 t-test 0.447653755485 0.654905142568
resonance_1.0_25 Cohen's d 0.0641355159363

resonance_1.0_40 t-test 0.746726289438 0.456219521472
resonance_1.0_40 Cohen's d 0.111661697526

resonance_0.2_10 t-test 1.03650354794 0.301176582912
resonance_0.2_10 Cohen's d 0.143342079545

resonance_0.2_25 t-test 0.240921984906 0.809871325157
resonance_0.2_25 Cohen's d 0.0344997697115

resonance_0.2_40 t-test 0.409553487155 0.682628727123
resonance_0.2_40 Cohen's d 0.0612078568973

resonance_0.05_10 t-test 2.64578422924 0.00877601427284
resonance_0.05_10 Cohen's d 0.366070994434

resonance_0.05_25 t-test 1.27400378963 0.204194072518
resonance_0.05_25 Cohen's d 0.182461084209

resonance_0.05_40 t-test 1.33192283316 0.184598079124
resonance_0.05_40 Cohen's d 0.199104019712

resonance_0.025_10 t-test 2.75672610718 0.00635984173369
resonance_0.025_10 Cohen's d 0.381395292855

resonance_0.025_25 [...]<jupyter_text>okay, whoaThat's a huge blinking neon light saying "publish me."Bestsellers are *at least* as influential than the canon. There's not even a lot of statistically significant evidence that the canon is influential at all, except when you look at "close resonance" in the 5% and 2.5% fraction.This result needs to be examined carefully. The power of bestsellerdom varies by time (see below), and the power of canonicity will be stronger for Norton proper than for Heath and Norton Short Fiction. Also, the significance gap is partly caused by a big difference in *N*: we have a lot more bestsellers than canonized titles in our metadata. (Roughly 800 vs 100.)But I think even after slicing and dicing the result we're going to find that we can say something like "If you want to find nineteenth-century novels that were ahead of their time, you're better off looking at bestsellers, or volumes well-reviewed in the nineteenth century, than at a list of titles currently assigned in college courses on the period.""Better off" might turn out to be an exaggeration, but at a minimum I suspect we can say "you'll do at least as well," and that's still a huge thing to be able to say. what about the specific titles we preregistered?This was the fifth hypotheses we preregistered with OSF. We couldn't find all the titles in Hathi, but we found twenty.<jupyter_code>prereg, notprereg = main_and_contrast_set(['preregistered'], hypothesis_meta, zdata)
general_test(prereg, notprereg, zdata)<jupyter_output>resonance_1.0_10 t-test 3.98757765943 0.0003236210011
resonance_1.0_10 Cohen's d 1.3399071216

resonance_1.0_25 t-test 4.49654696207 8.06373095525e-05
resonance_1.0_25 Cohen's d 1.53893747919

resonance_1.0_40 t-test 5.42824605414 9.65907123498e-06
resonance_1.0_40 Cohen's d 2.04902702368

resonance_0.2_10 t-test 3.46369210984 0.0014249806473
resonance_0.2_10 Cohen's d 1.15907770706

resonance_0.2_25 t-test 4.40171217501 0.000106200848094
resonance_0.2_25 Cohen's d 1.49960896448

resonance_0.2_40 t-test 5.48505331687 8.29876166328e-06
resonance_0.2_40 Cohen's d 2.05527043677

resonance_0.05_10 t-test 2.80016001782 0.00825897888746
resonance_0.05_10 Cohen's d 0.937886502276

resonance_0.05_25 t-test 4.29729362594 0.000143637697755
resonance_0.05_25 Cohen's d 1.46136459099

resonance_0.05_40 t-test 5.49364860714 8.11045440775e-06
resonance_0.05_40 Cohen's d 2.06263158789

resonance_0.025_10 t-test 2.57331737562 0.0144687211737
resonance_0.025_10 Cohen's d 0.865807757845

resonance_0.025_[...]<jupyter_text>We can certainly pick em!!When we consciously aimed at things we expected to be "like the future," our aim was true. Statistically significant even at n = 20, and effect sizes are huge.That's fucking interesting.<jupyter_code>special = zdata.loc[prereg, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
special = special.join(meta.author, how = 'inner')
special<jupyter_output><empty_output><jupyter_text><NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, and <NAME> are the champions of this round.  Why is the "canon" so lame?At this point in the notebook, I switch from rigorous effect sizes and signficance levels to just measuring the mean z score for certain categories. The reason is just that I want to move quickly to compare a bunch of different subcategories.We can circle back to measure effect sizes more rigorously.<jupyter_code>norton, notnorton = main_and_contrast_set(['norton'], hypothesis_meta, zdata)
nortonvols = zdata.loc[norton, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
nortonvols = nortonvols.join(meta.author, how = 'inner')
nortonvols<jupyter_output><empty_output><jupyter_text>what's the mean for Norton?<jupyter_code>np.mean(nortonvols.loc[~np.isnan(nortonvols['resonance_0.05_10']), 'resonance_0.05_10'])
short, notshort = main_and_contrast_set(['nortonshort'], hypothesis_meta, zdata)
shortvols = zdata.loc[short, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
np.mean(shortvols.loc[~np.isnan(shortvols['resonance_0.05_10']), 'resonance_0.05_10'])<jupyter_output><empty_output><jupyter_text>aha — Norton short fiction is the canon-killing villain!<jupyter_code>heath, notheath = main_and_contrast_set(['heath'], hypothesis_meta, zdata)
heathvols = zdata.loc[heath, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
heathvols = heathvols.join(meta.author, how = 'inner')
heathvols
np.mean(heathvols.loc[~np.isnan(heathvols['resonance_0.05_10']), 'resonance_0.05_10'])<jupyter_output><empty_output><jupyter_text>Heath also brings down the averagebut compare the preregistered volumes:<jupyter_code>np.mean(special.loc[~np.isnan(special['resonance_0.05_10']), 'resonance_0.05_10'])<jupyter_output><empty_output><jupyter_text>The raw value in bestsellerdom is not huge (though a bit larger than Heath) but the n is large enough to make statistical significance very strong<jupyter_code>best, notbest = main_and_contrast_set(['bestseller'], hypothesis_meta, zdata)
bestvols = zdata.loc[best, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
bestvols = bestvols.join(meta.author, how = 'inner')
print('n = ', len(best))
np.mean(bestvols.loc[~np.isnan(bestvols['resonance_0.05_10']), 'resonance_0.05_10'])<jupyter_output>*
*
*
n =  832<jupyter_text>How does the power of bestsellerdom vary by time?<jupyter_code>def main_and_contrast_by_date(startdate, enddate, categorylist, hypmeta, data):
    vols_in_cat = []
    dates_of_cat = []
    for cat in categorylist:
        vols_in_cat.extend(hypmeta.loc[(hypmeta[cat] == True) & (hypmeta.latestcomp >= startdate) & (hypmeta.latestcomp < enddate), 'docid'])
        dates_of_cat.extend(hypmeta.loc[(hypmeta[cat] == True) & (hypmeta.latestcomp >= startdate) & (hypmeta.latestcomp < enddate), 'latestcomp'])
        
    contrast_vols = []
    for d in dates_of_cat:
        population = data.index[data['latestcomp'] == d].tolist()
        population = set(population) - set(vols_in_cat)
        if len(population) < 1:
            print('*')
            continue
        else:
            chosen = random.sample(population, 1)[0]
            contrast_vols.append(chosen)
    
    return vols_in_cat, contrast_vols
# 19c

best, notbest = main_and_contrast_by_date(1800, 1900, ['bestseller'], hypothesis_meta, zdata)
bestvols = zdata.loc[best, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
bestvols = bestvols.join(meta.author, how = 'inner')
print('n = ', len(best))
np.mean(bestvols.loc[~np.isnan(bestvols['resonance_0.05_10']), 'resonance_0.05_10'])<jupyter_output>n =  162<jupyter_text>as strong as our preregistered set!<jupyter_code># early 20c

best, notbest = main_and_contrast_by_date(1900, 1950, ['bestseller'], hypothesis_meta, zdata)
bestvols = zdata.loc[best, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
bestvols = bestvols.join(meta.author, how = 'inner')
print('n = ', len(best))
np.mean(bestvols.loc[~np.isnan(bestvols['resonance_0.05_10']), 'resonance_0.05_10'])<jupyter_output>n =  429<jupyter_text>weaker ....<jupyter_code># late 20c

best, notbest = main_and_contrast_by_date(1950, 2010, ['bestseller'], hypothesis_meta, zdata)
bestvols = zdata.loc[best, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
bestvols = bestvols.join(meta.author, how = 'inner')
print('n = ', len(best))
np.mean(bestvols.loc[~np.isnan(bestvols['resonance_0.05_10']), 'resonance_0.05_10'])<jupyter_output>n =  238<jupyter_text>nothing left.  Oh, what about mostdiscussed?<jupyter_code>buzzy, notbuzzy = main_and_contrast_set(['mostdiscussed'], hypothesis_meta, zdata)
general_test(buzzy, notbuzzy, zdata)
buzzyvols = zdata.loc[buzzy, ['resonance_0.025_25', 'resonance_1.0_40', 'resonance_0.05_10', 'latestcomp']]
buzzyvols = buzzyvols.join(meta.title, how = 'inner')
buzzyvols
np.mean(buzzyvols.loc[~np.isnan(buzzyvols['resonance_0.05_10']), 'resonance_0.05_10'])<jupyter_output><empty_output><jupyter_text>write out the results for easy analysis<jupyter_code>zdata.columns
zdata.to_csv('../supp2results/zdata.tsv', sep = '\t', index_label = 'docid')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09500415690424004, 'fraction_numerical': 0.11253873478950949, 'mean_word_length': 5.019790718835305, 'pattern_counts': {'":': 0, '<': 111, '<?xml version=': 0, '>': 109, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '27', 'dir': 'jupyter-structured-clean-dedup', 'id': '203071', 'license': 'MIT', 'n_tokens_mistral': 11016, 'n_tokens_neox': 8898, 'n_words': 2710, 'path': 'analysis/second_supplement_analysis1.ipynb', 'repo_name': 'tedunderwood/asymmetry', 'size': '26512'}