# NanoTranslator-immersive_translate-365M

[English](README.md) | 简体中文

## Introduction

NanoTranslator-immersive_translate-365M 是由 [NanoLM-365M-Base](https://huggingface.co/Mxode/NanoLM-365M-Base) 在 [wmt-19](https://huggingface.co/datasets/wmt/wmt19) 数据集上训练了 600 万数据得来的专门用于**中英双语**的翻译模型。

此模型遵循[沉浸式翻译](https://immersivetranslate.com/)（Immersive Translate）的 prompt 格式进行训练，可以通过 vllm、lmdeploy 等方式部署为 OpenAI 格式接口，从而完成调用。

## How to use

下面是一个用 transformers 调用的方式，prompt 遵循沉浸式翻译以保持最佳效果。

```python
import torch
from typing import Literal
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = 'Mxode/NanoTranslator-immersive_translate-365M'

model = AutoModelForCausalLM.from_pretrained(model_path).to('cuda:0', torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_path)

def translate(
    text: str,
    to: Literal["chinese", "english"] = "chinese",
    **kwargs
):
    generation_args = dict(
        max_new_tokens = kwargs.pop("max_new_tokens", 512),
        do_sample = kwargs.pop("do_sample", True),
        temperature = kwargs.pop("temperature", 0.35),
        top_p = kwargs.pop("top_p", 0.8),
        top_k = kwargs.pop("top_k", 40),
        **kwargs
    )

    prompt = """Translate the following source text to {to}. Output translation directly without any additional text.
    Source Text: {text}

    Translated Text:"""

    messages = [
        {"role": "system", "content": "You are a professional, authentic machine translation engine."},
        {"role": "user", "content": prompt.format(to=to, text=text)}
    ]
    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    model_inputs = tokenizer([inputs], return_tensors="pt").to(model.device)

    generated_ids = model.generate(model_inputs.input_ids, **generation_args)
    generated_ids = [
        output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
    ]

    response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
    return response

text = "After a long day at work, I love to unwind by cooking a nice dinner and watching my favorite TV series. It really helps me relax and recharge for the next day."
response = translate(text=text, to='chinese')
print(f'Translation: {response}')

"""
Translation: 工作了一天，我喜欢吃一顿美味的晚餐，看我最喜欢的电视剧，这样做有助于我放松，补充能量。
"""
```