---
language:
- en
- ko
metrics:
- bleu
base_model:
- google/gemma-2-9b-it
tags:
- translation
- korean
- colloquial
---

# gemma2_colloquial_korean_translator

## Model Description

This model is fine-tuned to translate English text into natural and fluent colloquial Korean based on the gemma-2-9b language model. It improves the accuracy and naturalness of translation by effectively reflecting expressions and vocabulary used in everyday conversation. It uses the PEFT (Parameter-Efficient Fine-Tuning) technique, specifically LoRA (Low-Rank Adaptation), for efficient training.

## Key Features

- Base Model: Google/gemma-2-9b-Instruct
- Task: English colloquial → Korean translation
- Training Technique: QLORA (Quantized Low-Rank Adaptation)
- Quantization: 4-bit quantization (nf4)
- LoRA Configuration:
  - rank (r): 6
  - alpha: 8
  - dropout: 0.05
  - target modules: "q_proj", "o_proj", "k_proj", "v_proj", "gate_proj", "up_proj", "down_proj"

## Training Data

The model was trained on a dataset consisting of English colloquial expressions and their corresponding Korean translations. The data was provided in JSON format.

(Specific) We used the 'Korean-English Translation Parallel Corpus for Daily Life and Colloquial Expressions' from AI Hub. This dataset includes 500,000 pairs of English-Korean text, significantly enhancing the model's ability to handle everyday expressions and colloquial language.

## Training Settings

- Epochs: 3
- Batch Size: 4
- Gradient Accumulation Steps: 4
- Learning Rate: 2e-4
- Weight Decay: 0.01
- Optimizer: AdamW (8-bit)
- Max Sequence Length: 512 tokens

## Usage

You can use this model to translate English colloquial expressions into Korean. Here's an example:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_path = "Soonchan/gemma2_colloquial_korean_translator"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path)

def translate(text):
    prompt = f"""<bos><start_of_turn>user
Please translate the following English colloquial expression into Korean.:
{text}<end_of_turn>
<start_of_turn>model
"""
    inputs = tokenizer(prompt, return_tensors="pt")
    outputs = model.generate(**inputs, max_new_tokens=100)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Usage example
english_text = "What's up?"
korean_translation = translate(english_text)
print(korean_translation)
```

## Limitations

- This model is specialized for colloquial expressions and may not be suitable for translating formal documents or technical content.
- The model's output should always be reviewed, as it may generate inappropriate or inaccurate translations depending on the context.

## License

This model follows the license of the original Gemma model. Please check the relevant license before use.

## References

- [Gemma: Open Models Based on Gemini Technology and Research](https://blog.google/technology/developers/gemma-open-models/)
- [PEFT: Parameter-Efficient Fine-Tuning of Billion-Scale Models on Low-Resource Hardware](https://arxiv.org/abs/2106.09685)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/abs/2305.14314)

## Framework Versions

- PEFT 0.12.0