README.md

---
license: gemma
datasets:
- lmg-anon/VNTL-v3.1-1k
- lmg-anon/VNTL-Chat
language:
- ja
- en
library_name: peft
base_model: google/gemma-2-27b
pipeline_tag: translation
---

# Summary

This is a [Gemma 2 27B](https://huggingface.co/google/gemma-2-27b) qlora trained using a slightly modified version of the [VNTL-v3.1-1k](https://huggingface.co/datasets/lmg-anon/VNTL-v3.1-1k) dataset, concatenated with the [VNTL-Chat](https://huggingface.co/datasets/lmg-anon/VNTL-Chat) dataset. 

This was trained mostly with the same hyperparameters as the [VNTL LLaMA 3 8B qlora](https://huggingface.co/lmg-anon/vntl-llama3-8b-qlora), the differences are:
- Changed batch size and gradient accumulation from 6/5 to 15/2, this was done mostly for training efficiency since this qlora was trained using an A100.

As the 8B qlora, this version also includes the "chat mode" for Japanese grammar questions.

## Translation Prompt

This is a prompt example for translation:
```
<<METADATA>>
[character] Name: Uryuu Sakuno (瓜生 桜乃) | Gender: Female
[character] Name: Uryuu Shingo (瓜生 新吾) | Gender: Male | Aliases: Onii-chan (お兄ちゃん)
<<TRANSLATE>>
<<JAPANESE>>
買い物に行ったっきりなかなか帰ってこないと思ったら、やっぱり、迷子になっていたらしい。
<<ENGLISH>>
I thought she was taking too long to come back from shopping, and it turns out she had gotten lost as I expected.<eos>
<<JAPANESE>>
[新吾]: 「そっか。うん、電話してみてよかった」
<<ENGLISH>>
[Shingo]: "Is that so? Well, I'm glad I called you then."<eos>
<<JAPANESE>>
彼女は基本的に落ち着いた頭のいい子なんだけど、結構な筋金の入った方向音痴でもある。
<<ENGLISH>>
She's basically a smart and level-headed girl, but she also has a very bad sense of direction.<eos>
<<JAPANESE>>
頻度こそさほどではないものの、ひとたび迷子になると、とにかく派手に道に迷ってしまう。
<<ENGLISH>>
Though not very frequently, once she gets lost, she gets lost in a spectacular way.<eos>
<<JAPANESE>>
[桜乃]: 『……ごめん』
<<ENGLISH>>
[Sakuno]: "... Sorry."<eos>
<<JAPANESE>>
[新吾]: 「ううん、こう言っちゃなんだけど、迷子でよかったよ。 桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」
<<ENGLISH>>
```

The generated translation for that prompt, with temperature 0, is:
```
[Shingo]: "No, it's fine. I'm glad you got lost, actually. You're so cute, I was worried about you."
```

## Chat Prompt

This is a prompt example for chat:
```
<<CHAT>>
<<HUMAN>>
Hi! Can you break down this sentence for me? Like, what is the meaning of "桜乃は可愛いから"?
"ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」"
<<LLM>>
```

The generated response for that prompt, with temperature 0, is:
```
Certainly! Let's start with your first question. "桜乃は可愛いから" is a phrase that consists of three parts:
1. "桜乃" is a proper noun, likely a person's name.
2. "は" is a particle that marks the topic of the sentence, in this case, "桜乃."
3. "可愛い" is an adjective that means "cute" or "adorable."
4. "から" is a conjunction that indicates a reason or cause.
So, the literal translation of "桜乃は可愛いから" would be "Because Sakura is cute." It suggests that the speaker is about to provide a reason or explanation based on the fact that the person named Sakura is cute.
```