<jupyter_start><jupyter_text>Analysis of the results<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
import pandas as pd


import myscripts.helper as H
from myscripts.myplot import *
# matplotlib settings
%matplotlib inline
plt.style.use('/Users/sst/visual.mplstyle')
# pandas settings
pd.set_option('float_format', '{:.2e}'.format)<jupyter_output><empty_output><jupyter_text>Data information<jupyter_code>DATA = pd.read_csv('meta/old_data.csv')
DATA<jupyter_output><empty_output><jupyter_text>Characteristic functionsWhat characteristic functions are the best? What do them tell?<jupyter_code>CSV = pd.read_csv('meta/csv.csv').loc[85:]
FGR = pd.read_csv('meta/fgr.csv').loc[85:]

FGR['c'] = DATA.set_index('sample').loc[FGR['name'], 'c'].to_list()
index = pd.DataFrame({
    'Two SphericalCFs': [85, 86, 94, 87, 95],  # Two SphericalCFs
    'SpheroidalCF': [88, 89, 96, 90, 97],  # SpheroidalCF
    'LognormalSphericalCF': [91, 92, 98, 93, 99],  # LognormalSphericalCF
}, dtype=int)<jupyter_output><empty_output><jupyter_text>Fitted PDFsAll three characteristic functions are indistinguishable by fitted PDFs. The $R_w$ and residuals are basically the same. All functions fit the data well.<jupyter_code>plt.figure(figsize=(24, 16))

ax = plt.subplot(131)
fitting = index.iloc[:, 0]
fgr = FGR.loc[fitting]
plot_fgr(fgr['file'], names=fgr['name'], colors=fgr['c'], normal=True, auto_rw=True)
plt.title(fitting.name)

plt.subplot(132, sharey=ax)
fitting = index.iloc[:, 1]
fgr = FGR.loc[fitting]
plot_fgr(fgr['file'], names=fgr['name'], colors=fgr['c'], normal=True, auto_rw=True)
plt.title(fitting.name)

plt.subplot(133, sharey=ax)
fitting = index.iloc[:, 2]
fgr = FGR.loc[fitting]
plot_fgr(fgr['file'], names=fgr['name'], colors=fgr['c'], normal=True, auto_rw=True)
plt.title(fitting.name)

plt.show()<jupyter_output><empty_output><jupyter_text>Fitting Results2Spherical: Nonthing to say.Spheroidal: The JBNP31 prefers the polar direction while the others tend to be longer along equator direction.Lognormal: The average particle size is small.<jupyter_code>res_dct = {}
num_rows = [4, 3, 3]
for (cf, index), num_row in zip(index.iteritems(), num_rows):
    csv = CSV.loc[index]
    res = H.join_result_with_std(csv['file'], column_names=csv['name'])
    res = res.iloc[2:]
    res_dct[cf] = res
    print(res.head(num_row).to_string(col_space=16), end='\n\n')<jupyter_output>JBNP31           JBNP32          JBNP32L           JBNP33          JBNP33L
frac_b1                 0.7+/-0.4        0.5+/-0.4        0.4+/-1.1      0.71+/-0.32        0.7+/-0.6
psize_Bronze_1            46+/-14          43+/-18      (4+/-5)e+01          52+/-14          53+/-23
psize_Bronze_2            21+/-16          19+/-10          22+/-23          21+/-17          24+/-28
scale_Bronze          0.54+/-0.09      0.53+/-0.11      0.30+/-0.14      0.52+/-0.09      0.39+/-0.09

                           JBNP31           JBNP32          JBNP32L           JBNP33          JBNP33L
erad_Bronze            14.6+/-3.5          29+/-13          26+/-21          38+/-22          37+/-28
prad_Bronze       (0.7+/-2.9)e+02            8+/-4            8+/-8           12+/-5           12+/-7
scale_Bronze          0.52+/-0.07      0.53+/-0.11      0.30+/-0.16      0.50+/-0.07      0.39+/-0.07

                           JBNP31           JBNP32          JBNP32L           JB[...]
{'fraction_non_alphanumeric': 0.15450771055753262, 'fraction_numerical': 0.07354685646500593, 'mean_word_length': 2.658351409978308, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '212862', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1462, 'n_tokens_neox': 1221, 'n_words': 243, 'path': 'analysis.ipynb', 'repo_name': 'st3107/19st_tio2b_notebooks', 'size': '3372'}