<jupyter_start><jupyter_text>CSV DATA EXTRACTION NOTEBOOKThis notebook shows the original dataset data.The original data has been transformed using operations from the pandas library into a more readable form. The data was then written to csv file.***This notebook is created for presentation purposes, the script ../src/csv_data_extraction.py is used to retrieve the data from the dataset and return it in a readable form***<jupyter_code>import pandas as pd
file_path = '../data/hepatotoxicity_ALT_MACCSFP_ready_set.csv'
output_file_path = '../data/alt_maccsfp.csv'
data_from_csv = pd.read_csv(file_path, header=1)
data_from_csv<jupyter_output><empty_output><jupyter_text>Extract values for column 0 and delete records without values for column 0<jupyter_code>column_0_values = data_from_csv.iloc[:, 0].dropna().reset_index(drop=True)
column_0_values<jupyter_output><empty_output><jupyter_text>Extract fingerprints data and delete records without values for fingerprints<jupyter_code>results = data_from_csv.iloc[:, 1:].dropna().reset_index(drop=True)
results<jupyter_output><empty_output><jupyter_text>Insert column 0 to fingerprints<jupyter_code>results.insert(0, "ALT", column_0_values)
results<jupyter_output><empty_output><jupyter_text>Save the results in a csv file<jupyter_code>results.to_csv(output_file_path, index = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08657373440939105, 'fraction_numerical': 0.007336757153338224, 'mean_word_length': 8.406896551724138, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '532110', 'license': 'Apache-2.0', 'n_tokens_mistral': 448, 'n_tokens_neox': 432, 'n_words': 112, 'path': 'notebooks/csv_data_extraction.ipynb', 'repo_name': 'DawidSzczerba/umwpl2021-hepatoxicity-prediction', 'size': '1363'}