<commit_before># lstm-machine-translation
Implementation of Machine Translation via TensorFlow

## Overview

This code implements the "Sequence to Sequence Learning with Neural Networks" published by Google (https://arxiv.org/abs/1409.3215). The model is suitable for machine translation and is considered to be among the recent state-of-the art models used by Google for Machine Translation.

There are many improvements that can be added in data the data preprocessing part that can further lower the training error and accuracy, however the current model should be a good working prototype to begin with.

For convenience we don't use real translated data, but in this case have decided to translate a sentence to a new sentence with reversed words and word order (e.g. "how are you" -> "uoy era woh"). You can reference the original paper why this mimics quite well the complexity of learning how to translate a spoken language.

The model uses embedding vectors to store the words understandings.
"#" is used as a "GO" symbol.
"." is used as a "\<EOS>" symbol.

## Possible tunings

At the moment the model is using 4 unrollings, followed by a "GO" symbol.
The number of layers, max number of nodes and embedding size are easily tunable to test different configurations.

## Results

The current results were achieved on 12,000 steps of batch training with 4 unrollings. Experimentations with bigger epochs size might be interesting:

```
Average loss at step 11900 : 1.37952561259 learning rate: 0.103912
Minibatch perplexity: 37.87
Validation set perplexity: 41.58
[' defined self by label # lebal yb fles denifed']
```<commit_msg>Update README with Requirements info<commit_after># lstm-machine-translation
Implementation of Machine Translation via TensorFlow

## Overview

This code implements the "Sequence to Sequence Learning with Neural Networks" published by Google (https://arxiv.org/abs/1409.3215). The model is suitable for machine translation and is considered to be among the recent state-of-the art models used by Google for Machine Translation.

There are many improvements that can be added in data the data preprocessing part that can further lower the training error and accuracy, however the current model should be a good working prototype to begin with.

For convenience we don't use real translated data, but in this case have decided to translate a sentence to a new sentence with reversed words and word order (e.g. "how are you" -> "uoy era woh"). You can reference the original paper why this mimics quite well the complexity of learning how to translate a spoken language.

The model uses embedding vectors to store the words understandings.
"#" is used as a "GO" symbol.
"." is used as a "\<EOS>" symbol.

## Possible tunings

At the moment the model is using 4 unrollings, followed by a "GO" symbol.
The number of layers, max number of nodes and embedding size are easily tunable to test different configurations.

## Results

The current results were achieved on 12,000 steps of batch training with 4 unrollings. Experimentations with bigger epochs size might be interesting:

```
Average loss at step 11900 : 1.37952561259 learning rate: 0.103912
Minibatch perplexity: 37.87
Validation set perplexity: 41.58
[' defined self by label # lebal yb fles denifed']
```

## Requirements
    python 2.7
    tensorflow 0.6.0

Tensorflow can be installed using

    export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.6.0-cp27-none-linux_x86_64.whl
    pip install --ignore-installed --upgrade $TF_BINARY_URL
{'fraction_non_alphanumeric': 0.05659848697114037, 'fraction_numerical': 0.030260577192490895, 'mean_word_length': 4.483870967741935, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 7, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2026817', 'n_tokens_mistral': 999, 'n_tokens_neox': 885, 'n_words': 507}