<jupyter_start><jupyter_text>Polymer Electrolyte Data Mining Project:  Step 1: Cleaning data file in preparation for visualizations and further statistical learning methods<jupyter_code>from rdkit import Chem
from mordred import Calculator, descriptors, Polarizability, HydrogenBond,RotatableBond,  VdwVolumeABC, Weight, AcidBase, AtomCount, ExtendedTopochemicalAtom, KappaShapeIndex  #GeometricalIndex,
import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>Import dataImport the data from excel as a dataframe, then inspect first 5 rows<jupyter_code># Import data from master excel file
data = pd.read_excel(r'C:\Users\nicol\Box Sync\Personal\Research\DataMining_PolymerElectrolyte_Review\DataSpreadsheets\Finalized_dataMining_Spreadsheet.xlsx')
data.tail()<jupyter_output><empty_output><jupyter_text>Look at dataframe characteristics:<jupyter_code>print(f'Dataframe size: {data.shape}')
print('Unique papers: ', len(pd.unique(data['DOI'])))
print('Unique polymers: ', len(pd.unique(data['Polymer'])))<jupyter_output>Dataframe size: (655, 75)
Unique papers:  65
Unique polymers:  78<jupyter_text>Preparing to calculate MORDRED descriptors for polymersIncludes drawing chemical structures for SMILES representations of molecules. Visualize results in stages so that images are large enough to analyze<jupyter_code># First, want to parse my data for each unique polymer (based on SMILES representation), so that I'm not recalculating
# the descriptors for each repeat of the same polymer...
SMILES_type1 = pd.unique(data['SMILES descriptor 1'])

# remove nan instances
SMILES_type_fix1 = SMILES_type1[~pd.isnull(SMILES_type1)]

# have to add Chem.MolFromSmiles for each string in list
SMILES_type_Chem1 = [Chem.MolFromSmiles(smile) for smile in SMILES_type_fix1]

print('Number of structures: ', len(SMILES_type_Chem1))
Chem.Draw.MolsToImage(SMILES_type_Chem1[0:10])
Chem.Draw.MolsToImage(SMILES_type_Chem1[10:20])
Chem.Draw.MolsToImage(SMILES_type_Chem1[20:30])
Chem.Draw.MolsToImage(SMILES_type_Chem1[30:37])
Chem.Draw.MolsToImage(SMILES_type_Chem1[37:44])
Chem.Draw.MolsToImage(SMILES_type_Chem1[44:48])
Chem.Draw.MolsToImage(SMILES_type_Chem1[48:55])
Chem.Draw.MolsToImage(SMILES_type_Chem1[55:62])
Chem.Draw.MolsToImage(SMILES_type_Chem1[62:66])
Chem.Draw.MolsToImage(SMILES_type_Chem1[66:])
# Repeat for comonomer 2
SMILES_type2 = pd.unique(data['SMILES descriptor 2'])

# remove nan instances
SMILES_type_fix2 = SMILES_type2[~pd.isnull(SMILES_type2)]

# have to add Chem.MolFromSmiles for each string in list
SMILES_type_Chem2 = [Chem.MolFromSmiles(smile) for smile in SMILES_type_fix2]

print('Number of structures: ', len(SMILES_type_Chem2))
Chem.Draw.MolsToImage(SMILES_type_Chem2[0:10])
Chem.Draw.MolsToImage(SMILES_type_Chem2[10:20])
Chem.Draw.MolsToImage(SMILES_type_Chem2[20:30])
Chem.Draw.MolsToImage(SMILES_type_Chem2[30:36])
Chem.Draw.MolsToImage(SMILES_type_Chem2[36:42])
Chem.Draw.MolsToImage(SMILES_type_Chem2[42:47])
Chem.Draw.MolsToImage(SMILES_type_Chem2[47:53])
Chem.Draw.MolsToImage(SMILES_type_Chem2[53:58])
Chem.Draw.MolsToImage(SMILES_type_Chem2[58:64])
Chem.Draw.MolsToImage(SMILES_type_Chem2[64:])<jupyter_output><empty_output><jupyter_text>Calculating MORDRED descriptors for polymersCan either calculate all descriptors, or just calculate a subset (the ones deemed the most relevant)Paper reference: https://jcheminf.biomedcentral.com/articles/10.1186/s13321-018-0258-yList of possible descriptors: https://mordred-descriptor.github.io/documentation/master/api/modules.html<jupyter_code># step 1: calculate MORDRED descriptors for each SMILES type
# This can be done in parallel using MOREDRED calc.pandas(list of SMILES)

# descriptors of interest:
MRD_descrip = [Polarizability, HydrogenBond,RotatableBond, VdwVolumeABC, Weight, AcidBase,AtomCount, ExtendedTopochemicalAtom, KappaShapeIndex]

# initialize MORDRED calculator
calc = Calculator(MRD_descrip)

# calculates descriptors for each SMILES type in the list SMILES_type. Stores this as a pandas dataframe
comon1 = calc.pandas(SMILES_type_Chem1)
comon2 = calc.pandas(SMILES_type_Chem2)<jupyter_output>100%|██████████| 70/70 [00:03<00:00, 21.25it/s]
100%|██████████| 70/70 [00:02<00:00, 24.21it/s]<jupyter_text>Ensure no missing entries, rename headers for comonomer 1 vs 2, and correct for truncated monomers<jupyter_code>print('Number of null instances of smiles comonomer 1: ',pd.isnull(SMILES_type1).sum())
print('Number of null instances of smiles comonomer 2: ',pd.isnull(SMILES_type2).sum())
# rename headers for comonomer1 MORDRED descriptors
newcolumns = {}

for column in comon1:
    newcolumns[column] = 'Comonomer1 ' + column

comon1_correct = comon1.rename(columns = newcolumns)
comon1_correct<jupyter_output><empty_output><jupyter_text>For truncated monomers, visualize results and then remove incorrect features (nHBDon)<jupyter_code>comon1_correct[32:34]
Chem.Draw.MolsToImage(SMILES_type_Chem1[32:34])
Chem.Draw.MolToImage(SMILES_type_Chem1[36])
comon1_correct[36:37]
Chem.Draw.MolToImage(SMILES_type_Chem1[58])
comon1_correct[58:59]
comon1_correct.loc[32, 'Comonomer1 nHBDon'] = 1
comon1_correct.loc[32, 'Comonomer1 nHBDon'] = 0
comon1_correct.loc[36, 'Comonomer1 nHBDon'] = 0
comon1_correct.loc[58, 'Comonomer1 nHBDon'] = 0<jupyter_output><empty_output><jupyter_text>Repeat analysis and fixes for comonomer 2<jupyter_code># rename headers for comonomer2 MORDRED descriptors
newcolumns = {}

for column in comon2:
    newcolumns[column] = 'Comonomer2 ' + column

comon2_correct = comon2.rename(columns = newcolumns)
comon2_correct.head(10)
Chem.Draw.MolsToImage(SMILES_type_Chem2[34:38])
comon2_correct[34:38]
Chem.Draw.MolsToImage(SMILES_type_Chem2[39:41])
comon2_correct[39:41]
Chem.Draw.MolsToImage(SMILES_type_Chem2[61:62])
comon2_correct[61:62]
Chem.Draw.MolsToImage(SMILES_type_Chem2[63:64])
comon2_correct[63:64]
comon2_correct.loc[34, 'Comonomer2 nHBDon'] = 1
comon2_correct.loc[35, 'Comonomer2 nHBDon'] = 0
comon2_correct.loc[36, 'Comonomer2 nHBDon'] = 0
comon2_correct.loc[37, 'Comonomer2 nHBDon'] = 0

comon2_correct.loc[39, 'Comonomer2 nHBDon'] = 0

comon2_correct.loc[40, 'Comonomer2 nHBDon'] = 0
comon2_correct.loc[40, 'Comonomer2 nAcid'] = 0

comon2_correct.loc[61, 'Comonomer2 nHBDon'] = 0

comon2_correct.loc[63, 'Comonomer2 nHBDon'] = 0
comon2_correct.loc[63, 'Comonomer2 nAcid'] = 0<jupyter_output><empty_output><jupyter_text>Add those descriptors to the relevant rows of the original dataframe<jupyter_code># first step, copy rows in result dataframe to create dataframe that's same size as data dataframe
Mordred_Columns1 = pd.DataFrame()
Mordred_Columns2 = pd.DataFrame()
listMordred1 = []
listMordred2 = []

for ind in data.index:
    for i in range(len(SMILES_type_fix1)):
        if data['SMILES descriptor 1'][ind] == SMILES_type_fix1[i]:
            listMordred1.append(comon1_correct.iloc[i])

for ind in data.index:
    for i in range(len(SMILES_type_fix2)):
        if data['SMILES descriptor 2'][ind] == SMILES_type_fix2[i]:
            listMordred2.append(comon2_correct.iloc[i])

Mordred_Columns1 = Mordred_Columns1.append(listMordred1, ignore_index = True)
Mordred_Columns2 = Mordred_Columns2.append(listMordred2, ignore_index = True)

Mordred_Columns1.shape
# then, concatenate the Mordred columns dataframe to the data dataframe
data_Mordred = pd.concat([data,Mordred_Columns1,Mordred_Columns2], axis = 1)<jupyter_output><empty_output><jupyter_text>Seem to be having issues with Kier3, so will drop those columns for both comonomers<jupyter_code>data_Mordred.drop(columns = ['Comonomer1 Kier3','Comonomer2 Kier3'], inplace = True)<jupyter_output><empty_output><jupyter_text>Calculate MORDRED descriptors for anion as well<jupyter_code>pd.unique(data_Mordred['Anion'])
# dictionary with anion and SMILES representation
anion_smiles = {'TFSI':'C(F)(F)(F)S(=O)(=O)[N-]S(=O)(=O)C(F)(F)F','CF3SO3':'C(F)(F)(F)S(=O)(=O)[O-]',
                'N(SO2C2F5)2':'C(F)(F)(F)C(F)(F)S(=O)(=O)[N-]S(=O)(=O)C(F)(F)C(F)(F)F','BF4':'[B-](F)(F)(F)F',
                'ClO4':'[O-]Cl(=O)(=O)=O','PF6':'F[P-](F)(F)(F)(F)F','AsF6':'F[As-](F)(F)(F)(F)F','I':'[I-]',
                'AlCl4':'[Al-](Cl)(Cl)(Cl)Cl','FSI':'FS(=O)(=O)[N-]S(=O)(=O)F',
                'MPSA':'C(F)(F)(F)S(=O)(=O)[N-]CCCOC','SCN':'N#C[S-]'}

AnionSmiles = []

for anion in data_Mordred['Anion']:
    AnionSmiles.append(anion_smiles[anion])
    
data_Mordred['Anion Smiles'] = AnionSmiles

Anion_SMILES_type = pd.unique(data_Mordred['Anion Smiles'])

# remove nan instances
Anion_SMILES_type_fix = Anion_SMILES_type[~pd.isnull(Anion_SMILES_type)]

# have to add Chem.MolFromSmiles for each string in list
Anion_SMILES_type_Chem = [Chem.MolFromSmiles(smile) for smile in Anion_SMILES_type_fix]

# plot the chemical structures to make sure I got them right
Chem.Draw.MolsToImage(Anion_SMILES_type_Chem)
# calculate MORDRED descriptors for anions

result_anion = calc.pandas(Anion_SMILES_type_Chem)<jupyter_output>100%|██████████| 12/12 [00:02<00:00,  5.41it/s]<jupyter_text>Rename headers for anion descriptors<jupyter_code># rename headers for anion MORDRED descriptors
newcolumns = {}

for column in result_anion:
    newcolumns[column] = 'anion ' + column

result_anion_corr = result_anion.rename(columns = newcolumns)
result_anion_corr<jupyter_output><empty_output><jupyter_text>Need to edit PF6 values because of high bonding environment<jupyter_code>pf6_failures = ['anion ETA_eta_R','anion AETA_eta_R', 'anion ETA_eta_RL', 'anion AETA_eta_RL', 'anion ETA_eta_F', 
                'anion AETA_eta_F','anion ETA_eta_FL', 'anion AETA_eta_FL', 'anion ETA_eta_B', 'anion AETA_eta_B',
                'anion ETA_eta_BR', 'anion AETA_eta_BR', 'anion ETA_dAlpha_A', 'anion ETA_dAlpha_B', 
                'anion ETA_epsilon_3','anion ETA_dEpsilon_A', 'anion ETA_dEpsilon_C']
for failure in pf6_failures:
    result_anion_corr.loc[5,failure] = np.mean([result_anion_corr.loc[0,failure],result_anion_corr.loc[1,failure],
                                                result_anion_corr.loc[2,failure],result_anion_corr.loc[3,failure],
                                               result_anion_corr.loc[4,failure],result_anion_corr.loc[6,failure],
                                               result_anion_corr.loc[10,failure],result_anion_corr.loc[11,failure],])<jupyter_output><empty_output><jupyter_text>Drop anion Kier3<jupyter_code>result_anion_corr.drop(columns = ['anion Kier3'],inplace = True)<jupyter_output><empty_output><jupyter_text>Append descriptors to each relevant row of dataframe<jupyter_code>Mordred_Columns_anion = pd.DataFrame()
listMordredanion = []

for ind in data_Mordred.index:
    for i in range(len(Anion_SMILES_type_fix)):
        if data_Mordred['Anion Smiles'][ind] == Anion_SMILES_type_fix[i]:
            listMordredanion.append(result_anion_corr.iloc[i])
            break
        
Mordred_Columns_anion = Mordred_Columns_anion.append(listMordredanion, ignore_index = True)

# concatenate the Mordred columns dataframe to the data dataframe
data_Mordred_anion = pd.concat([data_Mordred,Mordred_Columns_anion], axis = 1)

#show dataframe size
data_Mordred_anion.shape<jupyter_output><empty_output><jupyter_text>Replace solvent with boiling point<jupyter_code>pd.unique(data_Mordred_anion['Solvent used'])
solvent_BP = {'acetonitrile':82,'dimethylcarbonate':90,'chloroform':61.2,
                 'acetone':56,'THF':66,'NMP':202,'none':-273.15,
                 'DMF':153, 'dimethoxyethane':185,'dimethyl ketone': 56, 'DMSO': 189, 'water': 100, 
             'methanol': 65, 'benzene': 80,}
BP_solvent = pd.DataFrame()
BP_solvent['solvent BP'] = [solvent_BP[solvent] for solvent in data_Mordred_anion['Solvent used']]

# concatenate the Mordred columns dataframe to the data dataframe
data_Mordred_anion = pd.concat([data_Mordred_anion,BP_solvent], axis = 1)

data_Mordred_anion<jupyter_output><empty_output><jupyter_text>Fill in any additionally missing dataThis includes salt concentrations (now that have MORDRED-calculated MW)<jupyter_code># filling in salt concentration data, if missing
for i in data_Mordred_anion.index:
    # first fill in li:monomer column for those samples where concentration given as salt wt%
    if pd.isnull(data_Mordred_anion['Li:functional group'][i]) and pd.isnull(data_Mordred_anion['salt wt%'][i]) == False:
        salt_mol = data_Mordred_anion['salt wt%'][i] / 100 / (data_Mordred_anion['anion MW'][i] + 6.941) #add in Li molar mass
        polymer_mw = data_Mordred['Comonomer1 MW'][i]*data_Mordred['Comonomer percentage'][i]/100 + data_Mordred['Comonomer2 MW'][i]*(100-data_Mordred['Comonomer percentage'][i])/100
        polymer_mol=(1 - data_Mordred_anion['salt wt%'][i] / 100) / polymer_mw
        data_Mordred_anion.loc[i,'Li:monomer'] = salt_mol / polymer_mol
    # then fill in Li:funcitonal group column for all samples that have Li:monomer column filled out
    if pd.isnull(data_Mordred_anion['Li:functional group'][i]) and pd.isnull(data_Mordred_anion['Li:monomer'][i]) == False:
        data_Mordred_anion.loc[i,'Li:functional group'] = data_Mordred_anion['Li:monomer'][i] / data_Mordred_anion['Average functional group per monomer'][i]<jupyter_output><empty_output><jupyter_text>Add any possible missing Tg data<jupyter_code># If Tg of polymer with salt is missing, BUT have Tg of polymer without salt, could use that as a proxy
# --> create new column 'approx Tg' for this
approxTg = []
for i in data_Mordred_anion.index:
    if pd.isnull(data_Mordred_anion['Tg'][i]) == False:
        approxTg.append(data_Mordred_anion['Tg'][i])
    elif pd.isnull(data_Mordred_anion['Tg'][i]) and pd.isnull(data_Mordred_anion['Tg polymer without salt'][i]) == False:
        approxTg.append(data_Mordred_anion['Tg polymer without salt'][i])        
    else:
        approxTg.append(np.nan)
data_Mordred_anion['approxTg'] = approxTg

data_Mordred_anion.head()<jupyter_output><empty_output><jupyter_text>Replace molecular weight with either Mn (if available) or Mw<jupyter_code>approxMW = []
for i in data_Mordred_anion.index:
    if pd.isnull(data_Mordred_anion['Polymer Mn (kDa)'][i]) == False:
        approxMW.append(data_Mordred_anion['Polymer Mn (kDa)'][i])
    elif pd.isnull(data_Mordred_anion['Polymer Mw (kDa)'][i]) == False:
        approxMW.append(data_Mordred_anion['Polymer Mw (kDa)'][i])
    else:
        approxMW.append(np.nan)
data_Mordred_anion['approxMW(kDa)'] = approxMW
data_Mordred_anion['approxMW(kDa)']<jupyter_output><empty_output><jupyter_text>Export data as csv for visualizationsAdd in index column which might be useful for visualizations<jupyter_code>data_Mordred_anion['Index'] = data_Mordred_anion.index<jupyter_output><empty_output><jupyter_text>Add https to doi for link<jupyter_code>DOILink = ["https://doi.org/" + str(i) for i in data_Mordred_anion['DOI']]

data_Mordred_anion['DOI'] = DOILink<jupyter_output><empty_output><jupyter_text>Save data to file<jupyter_code>file_name = 'Cleaned_Final_Data_6_2_2020.csv'
file_path = r'C:\Users\nicol\Box Sync\Personal\Research\DataMining_PolymerElectrolyte_Review\DataSpreadsheets\_' 
file = file_path + file_name
data_Mordred_anion.to_csv(file, sep=',',index = False)<jupyter_output><empty_output><jupyter_text>Clean data further for statistical learningWant to remove all columns that aren't populated enough, as well as those rows that contain NaNs<jupyter_code>dataML = data_Mordred_anion.copy()<jupyter_output><empty_output><jupyter_text>Make relevant columns into log format<jupyter_code>dataML['logCond60'] = np.log10(dataML['Conductivity at 60C'])
dataML['logCond30'] = np.log10(dataML['Conductivity at 30C'])
dataML['logCond90'] = np.log10(dataML['Conductivity at 90C'])
dataML['log Li:functional group'] = np.log10(dataML['Li:functional group'])<jupyter_output><empty_output><jupyter_text>Drop columns that I don't wantNote: don't yet drop polymer family, polymer or anion columns because I'll use them for the visualization process<jupyter_code>dataML.isna().sum()[0:50]
dataML.isna().sum()[50:100]
dataML.isna().sum()[100:150]
dataML.isna().sum()[150:200]
dataML.isna().sum()[200:250]
dataML.isna().sum()[250:300]
dataML.isna().sum()[300:]
# list columns to drop
dropCol = ['Conductivity at 0C', 'Conductivity at 15C', 'Conductivity at 20C',
       'Conductivity at 21C', 'Conductivity at 25C', 'Conductivity at 27C',
       'Conductivity at 30C', 'Conductivity at 35C', 'Conductivity at 40C',
       'Conductivity at 45C', 'Conductivity at 50C', 'Conductivity at 55C',
       'Conductivity at 60C', 'Conductivity at 65C', 'Conductivity at 70C',
       'Conductivity at 75C', 'Conductivity at 80C', 'Conductivity at 85C',
       'Conductivity at 90C', 'Conductivity at 100C', 'Conductivity at 110C',
       'Conductivity at 125C','salt wt%', 'Polymer system Notes', 
       'SMILES descriptor 1', 'SMILES descriptor 2', 'Li:monomer',
           'Tm', 'Tm start', 'Tm end', '% crystallinity','Transference number', 'T for transference',
       'Transference notes',
       'storage modulus', 'T for storage mod',
       'viscosity (mPa s)', 'Arrhenius Ea (eV) low T',
       'Arrhenius Ea (eV) high T', 'Arrhenius Ea (eV) low T me',
       'Arrhenius prefactor low T me (S/cm)','Arrhenius notes',
       'Li diffusion coefficient cm^2/s', 'temperature for D_Li',
       'X-ray structural data (aggregate? Distance?)',
           'Notes', 'DOI', 'Reference', 'Index','Li:functional group', 
          'logCond30', 'logCond90','youngs modulus (Mpa)','VFT Notes (above temp)',
           'Comonomer1 AMW','Comonomer2 AMW','anion AMW',
          'Average functional group per monomer',
          'PDI', 'Polymer Mn (kDa)',
       'Polymer Mw (kDa)','Tg', 'Tg polymer without salt',
          'VFT prefactor with set T0','VFT activation energy with fixed T0 (K)', 'Fixed T0 (degC)',
          'Anion Smiles','Solvent used','VFT prefactor (S/cm*T^(1/2))',
       'VFT activation energy (K)', 'VFT T0 (degC)',
       'Arrhenius Ea (eV) high T me', 'Arrhenius prefactor high T me (S/cm)',]

data_sparse = dataML.drop(columns = dropCol)
data_sparse.columns
data_sparse.isna().sum()[0:50]
data_sparse.isna().sum()[50:100]
data_sparse.isna().sum()[100:150]
data_sparse.isna().sum()[150:200]
data_sparse.isna().sum()[200:]<jupyter_output><empty_output><jupyter_text>Convert strings into integers or one-hot vectors  Convert crystallinity<jupyter_code>pd.unique(data_sparse['crystalline?'])
# Only have two values, yes or no, which should make this straightforward
data_sparse.loc[(data_sparse['crystalline?'] =='yes'),'crystalline?'] = 1
data_sparse.loc[(data_sparse['crystalline?'] =='no'),'crystalline?'] = 0
data_sparse.loc[(data_sparse['crystalline?'] =='na'),'crystalline?'] = -1

data_sparse.head()<jupyter_output><empty_output><jupyter_text>Convert drying vacuum into vectors as well<jupyter_code>pd.unique(data_sparse['drying vacuum'])
data_sparse.loc[(data_sparse['drying vacuum'] =='yes'),'drying vacuum'] = 2
data_sparse.loc[(data_sparse['drying vacuum'] =='none'),'drying vacuum'] = 0
data_sparse.loc[pd.isnull(data_sparse['drying vacuum']),'drying vacuum'] = 0
data_sparse.loc[(data_sparse['drying vacuum'] =='high'),'drying vacuum'] = 3
data_sparse.loc[(data_sparse['drying vacuum'] =='dry nitrogen'),'drying vacuum'] = 1<jupyter_output><empty_output><jupyter_text>For missing drying time/temp, change time to 8 h and temp to 25 C<jupyter_code>data_sparse.loc[pd.isnull(data_sparse['drying temp']),'drying temp'] = 25
data_sparse.loc[pd.isnull(data_sparse['drying time (h)']),'drying time (h)'] = 8
data_sparse.isna().sum()[0:10]
data_sparse.isna().sum()[-10:]<jupyter_output><empty_output><jupyter_text>Drop any remaining NaN rows before converting other strings to one-hot vectors<jupyter_code>data_sparse_noNaN = data_sparse.copy()
data_sparse_noNaN = data_sparse_noNaN.dropna()
data_sparse_noNaN.reset_index(drop=True, inplace = True)

print(f'DataFrame shape before dropping NaNs: {data_sparse.shape}')
print(f'DataFrame shape after dropping NaNs: {data_sparse_noNaN.shape}')<jupyter_output>DataFrame shape before dropping NaNs: (655, 236)
DataFrame shape after dropping NaNs: (271, 236)<jupyter_text>Convert chain architecture and solvent used into one-hot vectors using pandas get_dummies.Note: get_dummies does one-hot encoding, seemingly similar to sklearn's: https://queirozf.com/entries/one-hot-encoding-a-feature-on-a-pandas-dataframe-an-example<jupyter_code>data_sparse_noNaN = pd.get_dummies(data_sparse_noNaN,columns = ['chain architecture'])
print(f'DataFrame shape after one-hot encoding: {data_sparse_noNaN.shape}')<jupyter_output>DataFrame shape after one-hot encoding: (271, 237)<jupyter_text>Save to .csv file for statistical learning<jupyter_code>file_name = 'Cleaned_Final_Data-forML_6_2_2020.csv'
file_path = r'C:\Users\nicol\Box Sync\Personal\Research\DataMining_PolymerElectrolyte_Review\DataSpreadsheets\_' 
file = file_path + file_name
data_sparse_noNaN.to_csv(file, sep=',',index = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1296765253953844, 'fraction_numerical': 0.03172631277175211, 'mean_word_length': 6.354181307097681, 'pattern_counts': {'":': 0, '<': 123, '<?xml version=': 0, '>': 121, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '31', 'dir': 'jupyter-structured-clean-dedup', 'id': '529235', 'license': 'MIT', 'n_tokens_mistral': 8425, 'n_tokens_neox': 7719, 'n_words': 1502, 'path': 'Data_Cleaning.ipynb', 'repo_name': 'nschauser/Polymer_Electrolyte', 'size': '20929'}