add scb_mt_enth_2020
{'login': 'cstorm125', 'id': 15519308, 'node_id': 'MDQ6VXNlcjE1NTE5MzA4', 'avatar_url': 'https://avatars.githubusercontent.com/u/15519308?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/cstorm125', 'html_url': 'https://github.com/cstorm125', 'followers_url': 'https://api.github.com/users/cstorm125/followers', 'following_url': 'https://api.github.com/users/cstorm125/following{/other_user}', 'gists_url': 'https://api.github.com/users/cstorm125/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/cstorm125/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/cstorm125/subscriptions', 'organizations_url': 'https://api.github.com/users/cstorm125/orgs', 'repos_url': 'https://api.github.com/users/cstorm125/repos', 'events_url': 'https://api.github.com/users/cstorm125/events{/privacy}', 'received_events_url': 'https://api.github.com/users/cstorm125/received_events', 'type': 'User', 'site_admin': False}
[]
[]
## scb-mt-en-th-2020: A Large English-Thai Parallel Corpus

The primary objective of our work is to build a large-scale English-Thai dataset for machine translation.
We construct an English-Thai machine translation dataset with over 1 million segment pairs, curated from various sources,
namely news, Wikipedia articles, SMS messages, task-based dialogs, web-crawled data and government documents.
Methodology for gathering data, building parallel texts and removing noisy sentence pairs are presented in a reproducible manner.
We train machine translation models based on this dataset. Our models' performance are comparable to that of
Google Translation API (as of May 2020) for Thai-English and outperform Google when the Open Parallel Corpus (OPUS) is
included in the training data for both Thai-English and English-Thai translation.
The dataset, pre-trained models, and source code to reproduce our work are available for public use.
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/1034/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/1034', 'html_url': 'https://github.com/huggingface/datasets/pull/1034', 'diff_url': 'https://github.com/huggingface/datasets/pull/1034.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/1034.patch', 'merged_at': '2020-12-03T16:57:23Z'}