<commit_before>Requirements
============


MongoDB
https://www.mongodb.com/

RabbitMQ
https://www.rabbitmq.com
enable management 
plugin rabbitmq-plugins enable rabbitmq_management
visit http://localhost:15672/


Running
=======

Start mongoDB
Start rabbitMQ
#Start solr
#Start web
Start subs
#Start loader
#Start indexer
POST to subs
#poll web for sample
PUT to subs
#poll web for updated sample
#Stop web<commit_msg>Expanded rabbit instruction<commit_after>Requirements
============


MongoDB
https://www.mongodb.com/

RabbitMQ
https://www.rabbitmq.com
enable management plugin 
rabbitmq-plugins enable rabbitmq_management

"guest" user can only connect via localhost
By default, the guest user is prohibited from connecting to the broker remotely; it can 
only connect over a loopback interface (i.e. localhost). This applies both to AMQP and 
to any other protocols enabled via plugins. Any other users you create will not (by default) 
be restricted in this way. This is configured via the loopback_users item in the configuration file.

If you wish to allow the guest user to connect from a remote host, you should set 
the loopback_users configuration item to []. A complete rabbitmq.config which does 
this would look like:
[{rabbit, [{loopback_users, []}]}]


visit http://localhost:15672/


Running
=======

Start mongoDB
Start rabbitMQ
#Start solr
#Start web
Start subs
#Start loader
#Start indexer
POST to subs
#poll web for sample
PUT to subs
#poll web for updated sample
#Stop web
{'fraction_non_alphanumeric': 0.08299866131191433, 'fraction_numerical': 0.006693440428380187, 'mean_word_length': 4.862745098039215, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3125906', 'n_tokens_mistral': 454, 'n_tokens_neox': 413, 'n_words': 192}