<jupyter_start><jupyter_text>Create MagIC contributionThis notebook combines data from BRa and BRIC to create a MagIC contribution.<jupyter_code>import pmagpy.ipmag as ipmag
filenames=['BRa/measurements.txt','BRIC/measurements.txt']
outfile='combined_MagIC/measurements.txt'
ipmag.combine_magic(filenames,outfile)
filenames=['BRa/specimens.txt','BRIC/specimens.txt']
outfile='combined_MagIC/specimens.txt'
ipmag.combine_magic(filenames,outfile)
filenames=['BRa/samples.txt','BRIC/samples.txt']
outfile='combined_MagIC/samples.txt'
ipmag.combine_magic(filenames,outfile)
filenames=['BRa/sites.txt','BRIC/sites.txt']
outfile='combined_MagIC/sites.txt'
ipmag.combine_magic(filenames,outfile)
filenames=['BRa/locations.txt','BRIC/locations.txt']
outfile='combined_MagIC/locations.txt'
ipmag.combine_magic(filenames,outfile)
pd.read_csv()<jupyter_output><empty_output><jupyter_text>Manual changes- delete orientation_flag column in samples.txt- delete age_sigma column in sites.txt- Years Cal BP to Ma in sites.txt- add more sig figs to latitude and longitude for BRa (46.3852, -90.6337) and BRIC (46.3866, -90.6373) to sites.txt- update location to be a single location with lat/lon bounds associated with the above sites in locations.txtAfter these changes, there were still numerous validation errors in the location table which were fixed:*The locations table is missing required column "location_type". **The locations table is missing required column "geologic_classes".**The locations table is missing required column "lithologies". (Row: 1)**The locations table is missing required column "age" since columns "age_low","age_high" are empty. (Row: 1)**The locations table is missing required column "age_low" since column "age" is empty. (Row**The locations table is missing required column "age_high" since column "age" is empty. (Row: 1)**The locations table is missing required column "age_unit". (Row: 1)*The sites table is missing required column "geologic_classes". (Rows: 1-8)**ADDED**The sites table is missing required column "geologic_types". (Rows: 1-8)**ADDED — note this is "sample material types"**The sites table is missing required column "lithologies". (Rows: 1-8)**ADDED**The sites table column "criteria" value "ACCEPT" is not present in table "criteria" column "criterion". (Rows: 1-8)**DELETED COLUMN**The samples table is missing required column "result_quality". (Rows: 1-74)**ADDED**The specimens table column "experiments" value "LP-DIR-T" is not present in table "measurements" column "experiment". (Rows: 1-414)**DELETED THE EXPERIMENT COLUMN**The measurements table column "specimen" value "unknown" is not present in table "specimens" column "specimen". (Row: 1270)**DELETED ONE ROW WITHOUT DATA**<jupyter_code>ipmag.upload_magic?
ipmag.upload_magic(dir_path='./combined_MagIC/')<jupyter_output>-I- Removing old error files from /Users/Laurentia/0000_Github/2018_Red_Bed_Intraclasts/Data/combined_MagIC: locations_errors.txt, samples_errors.txt, specimens_errors.txt, sites_errors.txt, ages_errors.txt, measurements_errors.txt, criteria_errors.txt, contribution_errors.txt, images_errors.txt
-W- Column 'core_depth' isn't in samples table, skipping it
-W- Column 'composite_depth' isn't in samples table, skipping it
-W- Invalid or missing column names, could not propagate columns
-I- locations file successfully read in
-I- Validating locations
-I- No row errors found!
-I- appending locations data to /Users/Laurentia/0000_Github/2018_Red_Bed_Intraclasts/Data/combined_MagIC/upload.txt
-I- 1 records written to locations file
-I- locations written to  /Users/Laurentia/0000_Github/2018_Red_Bed_Intraclasts/Data/combined_MagIC/upload.txt
-I- measurements file successfully read in
-I- Validating measurements
-I- No row errors found!
-I- appending measurements data to /Users/Laurentia/0000_Gi[...]
{'fraction_non_alphanumeric': 0.105716523101018, 'fraction_numerical': 0.019838162359697206, 'mean_word_length': 6.383429672447013, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '79773', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1263, 'n_tokens_neox': 1176, 'n_words': 371, 'path': 'Data/create_MagIC.ipynb', 'repo_name': 'Swanson-Hysell-Group/2018_Red_Bed_Intraclasts', 'size': '3831'}