<commit_before>#Neural Monkey

[![Build Status](https://travis-ci.org/ufal/neuralmonkey.svg?branch=master)](https://travis-ci.org/ufal/neuralmonkey)

Neural Sequence Learning Using Tensorflow


### Usage

`python train.py <EXPERIMENT_INI>`

### Package Overview

The toolkit is organized into packages as follows:

- `encoders`: The encoders (get sequences of data and encode them)
- `decoders`: The decoders (get outputs of encoders and decode them)
- `trainers`: The trainers (train the whole thing)
- `runners`: The runners (run the trained thing)
- `readers`: The readers (read files)
- `processors`: The pre- and postprocessors (pre- and postprocess data)

### Other files

- `caffe_image_features.py` extracts features from images using pre-trained network
- `decoding_function.py` todo
- `install_caffe.sh` installs caffe including prerequisites
- `learning_utils.py` logging, header printing, training loop
- `precompute_image_features.py` deprecated
- `prepare_env.sh` self-explaining
- `prepare_vgg16.sh` deprecated
- `README.md` this file
- `reformat_downloaded_image_features.py` deprecated
- `tagging` see tagging/README.md
- `test_vocabulary.py` unit test for vocabulary
- `tokenize_data.py` tokenizes data<commit_msg>config package readme<commit_after>#Neural Monkey

[![Build Status](https://travis-ci.org/ufal/neuralmonkey.svg?branch=master)](https://travis-ci.org/ufal/neuralmonkey)

Neural Sequence Learning Using Tensorflow


### Usage

`python train.py <EXPERIMENT_INI>`

### Package Overview

The toolkit is organized into packages as follows:

- `encoders`: The encoders (get sequences of data and encode them)
- `decoders`: The decoders (get outputs of encoders and decode them)
- `trainers`: The trainers (train the whole thing)
- `runners`: The runners (run the trained thing)
- `readers`: The readers (read files)
- `processors`: The pre- and postprocessors (pre- and postprocess data)
- `config`: The configuration (loading, saving, parsing)

### Other files

- `caffe_image_features.py` extracts features from images using pre-trained network
- `decoding_function.py` todo
- `install_caffe.sh` installs caffe including prerequisites
- `learning_utils.py` logging, header printing, training loop
- `precompute_image_features.py` deprecated
- `prepare_env.sh` self-explaining
- `prepare_vgg16.sh` deprecated
- `README.md` this file
- `reformat_downloaded_image_features.py` deprecated
- `tagging` see tagging/README.md
- `test_vocabulary.py` unit test for vocabulary
- `tokenize_data.py` tokenizes data
{'fraction_non_alphanumeric': 0.11071569790431, 'fraction_numerical': 0.0015816528272044287, 'mean_word_length': 5.309226932668329, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1580987', 'n_tokens_mistral': 838, 'n_tokens_neox': 791, 'n_words': 239}