Splitting prepare_dataset into preparing the base dataset, and the tokenized dataset. This will help us to have further control over caching and loading data, eventually removing the storage of base dataset.

Updating NLTK requirements due to vulnerability in versions below 3.6.4: contained an inefficient Regular Expression and is vulnerable to regular expression denial of service attacks

Continuing cache minimizing in new repository. Please see https://github.com/huggingface/DataMeasurements for full history

:art: add line to file to bump ci

hate speech18 cache

hate speech 18 pmi file cache

Test to push up a simple cache file

:construction_worker: update CI to rebase

:bug: filter_vocab -> filter_words

:bug: really make sure log_files/ exists

:bug: add log_files dir if not exists

:rocket: add app

:rocket: add app and reqs

:construction_worker: add CI

:bug: remove line added by CoPilot

:memo: add README.md

:tada: init

Initial commit