<commit_before>sbt tasks
====================

We provide various sbt tasks to be run on the ADAMpro project, to simplify the deployment and development. For instance, go to the ADAMpro folder and run `sbt proto` to generate the protobuf file, or run `sbt buildDocker` to build a runnable docker image which contains ADAMpro.

*   `proto`: generates a jar file from the grpc folder and includes it in the main project (this is necessary, as shadowing is necessary of the netty dependency)

*   `setupDocker`: sets up the necessary docker containers for running ADAMpro using docker
*   `destroyDocker`: deletes the docker containers
*   `startDocker`: starts the docker containers
*   `stopDocker`: stops the docker containers without deleting them
*   `runDocker`: runs ADAMpro in docker container, performs first an assembly of and then does a sparkSubmit

*   `buildDocker`: build a self-contained docker ADAMpro container (adampro:latest) with all necessary packages installed<commit_msg>update documentation<commit_after>sbt tasks
====================

We provide various sbt tasks to be run on the ADAMpro project, to simplify the deployment and development. For instance, go to the ADAMpro folder and run `sbt proto` to generate the protobuf file, or run `sbt buildDocker` to build a runnable docker image which contains ADAMpro.

*   `proto`: generates a jar file from the grpc folder and includes it in the main project (this is necessary, as shadowing is necessary of the netty dependency)

*   `setupDocker`: sets up a self-contained Docker container for running ADAMpro

{'fraction_non_alphanumeric': 0.07079646017699115, 'fraction_numerical': 0.0, 'mean_word_length': 4.996212121212121, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2081820', 'n_tokens_mistral': 419, 'n_tokens_neox': 391, 'n_words': 221}