<jupyter_start><jupyter_text>Read in the data..<jupyter_code>data = pd.read_csv('/Users/Frankie/Documents/Dissertation/Data/pancreatic/24hProbeExpressionValues.csv')
data[:5]<jupyter_output><empty_output><jupyter_text>The columns are the instances and rows the features so we need to transpose the dataset.<jupyter_code>data = data.T<jupyter_output><empty_output><jupyter_text>Read in the labels...<jupyter_code>label = pd.read_csv('/Users/Frankie/Documents/Dissertation/Data/pancreatic/24hTargets.csv')
label[:5]<jupyter_output><empty_output><jupyter_text>We are using the OAC labeling...<jupyter_code>label = label[['FileName', 'OAC']]
label[:5]<jupyter_output><empty_output><jupyter_text>Join the data and labels on the FileName, remove any null rows and create a label column with 0s and 1s.<jupyter_code>joined_tables = label.join(data, on='FileName', how = 'outer')
joined_tables = joined_tables[pd.notnull(joined_tables['Probe1'])]
joined_tables['label'] = np.where(joined_tables['OAC']=='Mild', 0, 1)
joined_tables[:5]<jupyter_output><empty_output><jupyter_text>Drop the FileName and OAC columns and export as a CSV file.<jupyter_code>joined_tables.drop(['FileName','OAC'], axis=1).to_csv("/Users/Frankie/Desktop/pancreatic.csv",index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13917122752150118, 'fraction_numerical': 0.0109460516028147, 'mean_word_length': 9.158730158730158, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '544741', 'license': 'MIT', 'n_tokens_mistral': 476, 'n_tokens_neox': 459, 'n_words': 78, 'path': 'Scripts/Data-Preprocessing/Preprocessing Pancreatic Data.ipynb', 'repo_name': 'frankiecrouch/Bayesian-Decision-Trees', 'size': '1279'}