<commit_before># Antero ETL for transform

TODO but transform really is made just by calling remotely SQL stored procedures that do the heavy lifting. Stored procedures are called via jenkins job that calls Python script callsp.py with arguments.

Stored procedures are created and maintained via SQL scripts under `/db` directory.
<commit_msg>Minor update to see jenkins+github plugin in action.<commit_after># Antero ETL for transform

This readme is TODO but transform really is made just by calling remotely SQL stored procedures that do the heavy lifting. Stored procedures are called via jenkins job that calls Python script callsp.py with arguments.

Stored procedures are created and maintained via SQL scripts under `/db` directory.

{'fraction_non_alphanumeric': 0.03234501347708895, 'fraction_numerical': 0.0, 'mean_word_length': 4.991935483870968, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6676727', 'n_tokens_mistral': 176, 'n_tokens_neox': 173, 'n_words': 108}