<commit_before>wp_comments_classifier
======================

I wrote this utility because I was unable to find a working spam filter for wordpress.
It uses rdbi to connect to a variety of databases which also means that you need to install the approprivate database driver manually as I haven't specified them as package dependencies.
I.e. after you've determined what database you're using:
	gem install rdbi-driver-(mysql|postgresql|sqlite3)

Then create a configuration like the following:
	{
		"rdbi_driver": "MySQL",
		"classifier_debug": false,
		"host": "localhost",
		"username": "user",
		"password": "password",
		"database": "databasename"
	}

and run
	classify_wp_comments config.json<commit_msg>formatting<commit_after>wp_comments_classifier
======================

I wrote this utility because I was unable to find a working spam filter for wordpress.
It uses rdbi to connect to a variety of databases which also means that you need to install the approprivate database driver manually as I haven't specified them as package dependencies.
I.e. after you've determined what database you're using:

	gem install rdbi-driver-(mysql|postgresql|sqlite3)

Then create a configuration like the following:

	{
		"rdbi_driver": "MySQL",
		"classifier_debug": false,
		"host": "localhost",
		"username": "user",
		"password": "password",
		"database": "databasename"
	}

and run

	classify_wp_comments config.json
{'fraction_non_alphanumeric': 0.1071932299012694, 'fraction_numerical': 0.0014104372355430183, 'mean_word_length': 5.334821428571429, 'pattern_counts': {'":': 12, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3448882', 'n_tokens_mistral': 427, 'n_tokens_neox': 387, 'n_words': 163}