<commit_before>gomen-data
==========

My scripts for getting various data related to Malaysia's government into CSV format.

I'll be using Ruby extensively in this project since it's also part of my learning process.

If you just want the output, just go directly into the `output` directory for each project. Those output files will be updated whenever I have the time :)

@AmreeZaid<commit_msg>Fix ayat<commit_after>gomen-data
==========

My scripts for getting various data related to Malaysia's government into CSV format.

I'll be using Ruby extensively in this project (part of my learning process).

If you just want the output, just go directly into the `output` directory for each project. Those output files will be updated whenever I have time :)

@AmreeZaid
{'fraction_non_alphanumeric': 0.06892067620286085, 'fraction_numerical': 0.0, 'mean_word_length': 4.833333333333333, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1606564', 'n_tokens_mistral': 196, 'n_tokens_neox': 184, 'n_words': 112}