---
license: cc-by-nc-sa-4.0
language:
- ko
- en
metrics:
- bleu
pipeline_tag: text2text-generation
tags:
- nmt
- aihub
---

# KOEN-T5-SMALL-V0

This model is for Korean to English Machine Translator, which is based on T5-small architecture, but trained from scratch.

#### Code

The training code is from my lecture([LLM을 위한 김기현의 NLP EXPRESS](https://fastcampus.co.kr/data_online_nlpexpress)), which is published on [FastCampus](https://fastcampus.co.kr/). You can check the training code in this github [repo](https://github.com/kh-kim/nlp-express-practice).

#### Dataset

The training dataset for this model is mainly from [AI-Hub](https://www.aihub.or.kr/). The dataset consists of 11M parallel samples.

#### Tokenizer

I use Byte-level BPE tokenizer for both source and target language. Since it covers both languages, tokenizer vocab size is 60k.

#### Architecture

The model architecture is based on T5-small, which is popular encoder-decoder model architecture. Please, note that this model is trained from-scratch, not fine-tuned.

#### Evaluation

I conducted the evaluation with 5 different test sets. Following figure shows BLEU scores on each test set.

![](images/koen.png)

![](images/avg.png)

DEEPCL model is private version of this model, which is trained on much more data.

#### Contact

Kim Ki Hyun (nlp.with.deep.learning@gmail.com)