<jupyter_start><jupyter_text>These are some quick examples<jupyter_code>import utils # utility functions that deal with documents
import model # everything that has to do with the actual abstractions that do the predictions and labeling
import main # abstractions for running predictions on a list of entries<jupyter_output><empty_output><jupyter_text>Any util function that starts with entries takes a list of entries, some additional params and returns either a new list of data based on each entry or returns an object in specific cases.  What you'll probably want to do first is load a list of entries from a directory into memory as an object. You can just call:<jupyter_code>entries = utils.load_entries("./example_entries")<jupyter_output><empty_output><jupyter_text>In order to extract some informations, such the date and title, I hade to make some assumptions about how the file is formatted. The title is the first h1 header of the file and the date must be in the format:`Date: mmm, dd, yyyy`If your files don't have an h1 title nor a date in the above format, it won't be loaded. If your entries are exported from Notion, they just need a date field and a title and then they should be compatible right away. This is an example of how an entry object looks with a title, date and body field:<jupyter_code>entries[0]<jupyter_output><empty_output><jupyter_text>Here's the type of thing you can do with your list of entries:<jupyter_code># The amount of times a term appears throughout your entries
utils.entries_frequency_query(entries, "I", exact=True)


# The mean word length throughout all entries
utils.entries_mean_word_len(entries)
# A random sentence from an entry
utils.entries_random_sentence(entries)<jupyter_output><empty_output><jupyter_text>Most utility functions are self-explanatory, so what else can you do?<jupyter_code># Calculate emotions in a given entry
main.entry_generate_report(entries[2])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05370843989769821, 'fraction_numerical': 0.0020460358056265983, 'mean_word_length': 5.985714285714286, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '5202', 'license': 'MIT', 'n_tokens_mistral': 537, 'n_tokens_neox': 526, 'n_words': 258, 'path': 'src/examples.ipynb', 'repo_name': 'sarmentow/semasia', 'size': '1955'}