<commit_before># dockerized idb

## building the container

in the parent directory (which contains the Dockerfile) run:

	docker build -t idb .

## configuration

edit `idb.env.example` changing values where appropriate
and save it to `idb.env` .

## running with `docker-compose`

if you want to use a containerized database and redis,
you can use 

	docker-compose up -d 

in the parent directory. this will start containers for mysql, redis
and the idb. you'll have to initialize the database on first run,
do so by running

	docker-compose exec app bash -lc 'bundle exec rake db:schema:load'

after starting the containers.<commit_msg>more documentation about the container setup<commit_after># dockerized idb

this, together with `../Dockerfile` and `../docker-compose.yml`, facilitates
running the idb as container. the following containers are used:

- app: running the idb with apache & passenger
- sidekiq: same image as idb, different command run
- mysql: database, with ./mysql mounted for database persistance
  this location can be changed in `docker-compose.yml`
- redis: sidekiq requirement

## building the container

in the parent directory (which contains `Dockerfile`) run:

	docker build -t idb .

to build with another ruby version, change 

	ARG ruby_version=ruby-2.6.3

at the beginning of `Dockerfile` to something rvm understands. the idb
image is build from `debian:buster-slim`, as i've had problems getting
the passenger images to work. 

the container is created to use passenger with apache and rvm for installing
the right ruby versions. eventually rvm could be replaced by something
which is a better fit for production, but in a container environment this
should be ok for now.

## configuration

edit `idb.env.example` changing values where appropriate
and save it to `idb.env` .

## running with `docker-compose`

if you want to use a containerized database and redis,
you can use 

	docker-compose up -d 

in the parent directory. this will start containers for mysql, redis,
sidekiq and the idb. you'll have to initialize the database on first run,
do so by running

	docker-compose exec app bash -lc 'bundle exec rake db:schema:load'

after starting the containers.

any of the redis or mysql containers should be replaceable by
modifying `idb.env` to use other systems. the only requirements
are the sidekiq and app containers.

{'fraction_non_alphanumeric': 0.05954391891891892, 'fraction_numerical': 0.001266891891891892, 'mean_word_length': 4.229580573951435, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3550497', 'n_tokens_mistral': 683, 'n_tokens_neox': 639, 'n_words': 337}