<commit_before># Steam Data Setup

This subdir is for gathering the data we need using freely available API's
with Python.

You should have Python 3 and `make` installed to use the `Makefile`<commit_msg>Add our current todo list to README's<commit_after># Steam Data Setup

This subdir is for gathering the data we need using freely available API's
with Python.

You should have Python 3 and `make` installed to use the `Makefile`

NOTE that `make clean` will delete the Python virtual env and all previous
data, so be sure that's what you want.

## In Progress

TODO: add app id and name from csv file to record - including for success=false
TODO: games.py should pre-scan games.json, only do missing appid's, and then append
TODO: above todo means our makefile would be incomplete
{'fraction_non_alphanumeric': 0.049872122762148335, 'fraction_numerical': 0.0025575447570332483, 'mean_word_length': 4.553191489361702, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6667308', 'n_tokens_mistral': 216, 'n_tokens_neox': 205, 'n_words': 125}