<commit_before>FrequentFeedScraper
===================

Frequently scrapes a feed that has few entries at a time, but updates frequently and creates a new feed with much more unique entires.
<commit_msg>Added some information to the readme<commit_after>FrequentFeedScraper
===================

Frequently scrapes a feed that has few entries at a time, but updates frequently and creates a new feed with much more unique entires.

Currently hard coded to scrape codereviews.chromium.org (it provides a twenty entry feed, but entries are added every several seconds during work hours, so feed readers such as Feedly cannot keep up because they are scraping it once in two hours at most).

{'fraction_non_alphanumeric': 0.08017492711370262, 'fraction_numerical': 0.0, 'mean_word_length': 5.361111111111111, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3122237', 'n_tokens_mistral': 170, 'n_tokens_neox': 158, 'n_words': 98}