<jupyter_start><jupyter_text>Data and GPT path checker     Version: 1.0    Date:    12/03/2019    Author:  <NAME> and <NAME> (Plymouth Marine Laboratory)    Credit:  This code was developed for EUMETSAT under contracts for the Copernicus              programme.    License: This code is offered as open source and free-to-use in the public domain,              with no warranty. To run the training python scripts, the code needs to know the path to your data and gpt executable (e.g. where they are stored on your computer). There are subtle differences in how paths are constructed in Windows, Linux and on OSx, so we have provided this quick tool to help you constructing reliable, cross-platform supported paths in python. To use this tool, you need to provide it with the actual path to your data as your computer provides it. The easiest way to get this is to navigate to the folder containing the training data folders using your the file manager for your OS, right click on the folder and select either 'properties' or 'get info' from the drop-down list. The path will be displayed on the page that opens, and will look something like C:\dir1\dir2\dir3 (where dir* indicates a directory).To run the script, click the "kernel" option and choose "restart and run all". This will create a white text box below. Copy your path into the text box and hit "Enter" to produce output that will help in running later scripts.To run the GPT path checker, you need to locate the gpt executable file (note here we give the file, not the directory as we did in MYPATH!). This would usually be here C:\Program Files\snap\bin\gpt.exe on Windows.<jupyter_code>import widget_functions as wf
import os
# Data path checker....
wf.text_box('data')
# GPT path checker....
wf.text_box('GPT')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.055924695459579184, 'fraction_numerical': 0.007198228128460687, 'mean_word_length': 3.9371584699453552, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '569458', 'license': 'MIT', 'n_tokens_mistral': 471, 'n_tokens_neox': 457, 'n_words': 278, 'path': 'Configuration_testing/Data_path_checker.ipynb', 'repo_name': 'africamarine-eo/training', 'size': '1822'}