Initial commit

README.md

@@ -0,0 +1,61 @@
+---
+language:
+- en
+- pl
+
+tags:
+- translation
+
+license: apache-2.0
+---
+
+### OPUS Tatoeba English-Polish
+
+*This model was obtained by running the script [convert_marian_to_pytorch.py](https://github.com/huggingface/transformers/blob/master/src/transformers/models/marian/convert_marian_to_pytorch.py) with the flag `-m eng-pol`. The original models were trained by [Jörg Tiedemann](https://blogs.helsinki.fi/tiedeman/) using the [MarianNMT](https://marian-nmt.github.io/) library. See all available `MarianMTModel` models on the profile of the [Helsinki NLP](https://huggingface.co/Helsinki-NLP) group.
+
+* source language name: English
+* target language name: Polish
+* OPUS readme: [README.md](https://object.pouta.csc.fi/Tatoeba-MT-models/eng-pol/README.md)
+
+* model: transformer
+* source language code: en
+* target language code: pl
+* dataset: opus 
+* release date: 2021-02-19
+* pre-processing: normalization + SentencePiece (spm32k,spm32k)
+* download original weights: [opus-2021-02-19.zip](https://object.pouta.csc.fi/Tatoeba-MT-models/eng-pol/opus-2021-02-19.zip/eng-pol/opus-2021-02-19.zip)
+* Training data: 
+  * eng-pol: Tatoeba-train (59742979)
+* Validation data: 
+  * eng-pol: Tatoeba-dev, 44146
+  * total-size-shuffled: 44145
+  * devset-selected: top 5000  lines of Tatoeba-dev.src.shuffled!
+* Test data: 
+  * Tatoeba-test.eng-pol: 10000/64925
+* test set translations file: [test.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/eng-pol/opus-2021-02-19.zip/eng-pol/opus-2021-02-19.test.txt)
+* test set scores file: [eval.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/eng-pol/opus-2021-02-19.zip/eng-pol/opus-2021-02-19.eval.txt)
+* BLEU-scores
+|Test set|score|
+|---|---|
+|Tatoeba-test.eng-pol|47.5|
+* chr-F-scores
+|Test set|score|
+|---|---|
+|Tatoeba-test.eng-pol|0.673|
+
+### System Info: 
+* hf_name: eng-pol
+* source_languages: en
+* target_languages: pl
+* opus_readme_url: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-pol/opus-2021-02-19.zip/README.md
+* original_repo: Tatoeba-Challenge
+* tags: ['translation']
+* languages: ['en', 'pl']
+* src_constituents: ['eng']
+* tgt_constituents: ['pol']
+* src_multilingual: False
+* tgt_multilingual: False
+* helsinki_git_sha: 70b0a9621f054ef1d8ea81f7d55595d7f64d19ff
+* transformers_git_sha: 7c6cd0ac28f1b760ccb4d6e4761f13185d05d90b
+* port_machine: databox
+* port_time: 2021-10-18-15:11

config.json

@@ -0,0 +1,43 @@
+{
+  "activation_dropout": 0.0,
+  "activation_function": "swish",
+  "architectures": [
+    "MarianMTModel"
+  ],
+  "attention_dropout": 0.0,
+  "bad_words_ids": [
+    [
+      65000
+    ]
+  ],
+  "bos_token_id": 0,
+  "classifier_dropout": 0.0,
+  "d_model": 512,
+  "decoder_attention_heads": 8,
+  "decoder_ffn_dim": 2048,
+  "decoder_layerdrop": 0.0,
+  "decoder_layers": 6,
+  "decoder_start_token_id": 65000,
+  "dropout": 0.1,
+  "encoder_attention_heads": 8,
+  "encoder_ffn_dim": 2048,
+  "encoder_layerdrop": 0.0,
+  "encoder_layers": 6,
+  "eos_token_id": 0,
+  "forced_eos_token_id": 0,
+  "init_std": 0.02,
+  "is_encoder_decoder": true,
+  "max_length": 512,
+  "max_position_embeddings": 512,
+  "model_type": "marian",
+  "normalize_embedding": false,
+  "num_beams": 6,
+  "num_hidden_layers": 6,
+  "pad_token_id": 65000,
+  "scale_embedding": true,
+  "static_position_embeddings": true,
+  "torch_dtype": "float16",
+  "transformers_version": "4.11.3",
+  "use_cache": true,
+  "vocab_size": 65001
+}

metadata.json

@@ -0,0 +1 @@
+{"hf_name": "eng-pol", "source_languages": "en", "target_languages": "pl", "opus_readme_url": "https://object.pouta.csc.fi/Tatoeba-MT-models/eng-pol/opus-2021-02-19.zip/README.md", "original_repo": "Tatoeba-Challenge", "tags": ["translation"], "languages": ["en", "pl"], "src_constituents": ["eng"], "tgt_constituents": ["pol"], "src_multilingual": false, "tgt_multilingual": false, "helsinki_git_sha": "70b0a9621f054ef1d8ea81f7d55595d7f64d19ff", "transformers_git_sha": "7c6cd0ac28f1b760ccb4d6e4761f13185d05d90b", "port_machine": "databox", "port_time": "2021-10-18-15:11"}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b4c565a82a950902d1ef3c16c594c412e242d523ea6d8532036b422689afae8
+size 221610115

special_tokens_map.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"source_lang": "eng", "target_lang": "pol", "unk_token": "<unk>", "eos_token": "</s>", "pad_token": "<pad>", "model_max_length": 512, "sp_model_kwargs": {}, "special_tokens_map_file": null, "tokenizer_file": null, "name_or_path": "marian_ckpt/eng-pol", "tokenizer_class": "MarianTokenizer"}