ymcki commited on
Commit
74de57e
1 Parent(s): f847f47

Upload 9 files

Browse files
README.md CHANGED
@@ -3,8 +3,7 @@ base_model: google/gemma-2-2b-jpn-it
3
  language:
4
  - multilingual
5
  datasets:
6
- - mlabonne/harmless_alpaca
7
- - mlabonne/harmful_behaviors
8
  library_name: transformers
9
  license: gemma
10
  license_link: https://ai.google.dev/gemma/terms
@@ -40,13 +39,21 @@ described by mlabonne.
40
 
41
  Layer 17 of the original model was chosen for abliteration.
42
  I also created another layer 18 abliterated model for comparison.
43
- These two layers were chosen due to they both produce uncensored response
44
- after respective layer was abliterated.
45
 
46
- It is uploaded here to be evaluated by the Open LLM Leaderboard to see how brain damaged it
47
- is compared to the original model.
48
 
49
- ORPO fine tuning is currently underway to see if it can regain its sanity. You can play with this model first or wait until I am done with the fine tuning.
 
 
 
 
 
 
 
 
 
 
 
50
 
51
  ## Benchmark (100.0*raw scores only)
52
 
@@ -55,10 +62,13 @@ Click on the model name go to the raw score json generated by Open LLM Leaderboa
55
  | Model | Average | IFEval | BHH | Math Lv5 | GPQA | MUSR | MMLU-PRO |
56
  | ----- | ------- | ------ | ----|--------- | ---- | ---- | -------- |
57
  | [gemma-2-2b-jpn-it](https://huggingface.co/datasets/open-llm-leaderboard/results/blob/main/google/gemma-2-2b-jpn-it/results_2024-10-15T15-21-39.173019.json) | 30.82 | 54.11 | 41.43 | 0.0 | 27.52 | 37.17 | 24.67 |
 
 
 
58
  | [gemma-2-2b-jpn-it-abliterated-17](https://huggingface.co/datasets/open-llm-leaderboard/results/raw/main/ymcki/gemma-2-2b-jpn-it-abliterated-17/results_2024-10-18T15-18-46.821674.json) | 30.29 | 52.65 | 40.46 | 0.0 | 27.18 | 36.90 | 24.55 |
59
  | [gemma-2-2b-jpn-it-abliterated-18](https://huggingface.co/datasets/open-llm-leaderboard/results/raw/main/ymcki/gemma-2-2b-jpn-it-abliterated-18/results_2024-10-18T15-41-42.399571.json) | 30.61 | 53.02 | 40.96 | 0.0 | 27.35 | 37.30 | 25.05 |
60
 
61
- It is only slightly dumber than the original.
62
 
63
  ## How to run this model
64
 
@@ -67,7 +77,7 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
67
  import transformers
68
  import torch
69
 
70
- model_id = "gemma-2-2b-jpn-it-abliterated-17"
71
  dtype = torch.bfloat16
72
 
73
  tokenizer = AutoTokenizer.from_pretrained(model_id)
@@ -93,9 +103,11 @@ pip install -U "huggingface_hub[cli]"
93
  Then, you can target the specific file you want:
94
 
95
  ```
96
- huggingface-cli download ymcki/gemma-2-2b-jpn-it-abliterated-17 --include "*" --local-dir ./
97
  ```
98
 
99
  ## Credits
100
 
101
- Thank you mlabonne for describing his abliteration method.
 
 
 
3
  language:
4
  - multilingual
5
  datasets:
6
+ - mlabonne/orpo-dpo-mix-40k
 
7
  library_name: transformers
8
  license: gemma
9
  license_link: https://ai.google.dev/gemma/terms
 
39
 
40
  Layer 17 of the original model was chosen for abliteration.
41
  I also created another layer 18 abliterated model for comparison.
 
 
42
 
43
+ ORPO fine tuning was performed for eight epoches.
 
44
 
45
+ | Epoch | loss | eval_loss |
46
+ | ----- | ---- | --------- |
47
+ | 1 | 1.20152769684791564 | 1.0501047372817993 |
48
+ | 2 | 1.25755584239959716 | 1.0144596099853516 |
49
+ | 3 | 0.93099724054336543 | 0.9957754611968994 |
50
+ | 4 | 0.88664623498916623 | 0.9857067465782166 |
51
+ | 5 | 0.86961059570312504 | 1.0203918218612670 |
52
+ | 6 | 0.98065975904464630 | 0.9958684444427490 |
53
+ | 7 | 0.38512575328350068 | 0.9686505198478699 |
54
+ | 8 | 1.41178888082504270 | 0.9652527570724487 |
55
+
56
+ The fine tuned model is uploaded here to be evaluated by the Open LLM Leaderboard to see if the slightly brain damaged non-ORPO model can be healed. Again, the fine tuning method is also based on one described by [mlabonne](https://towardsdatascience.com/fine-tune-llama-3-with-orpo-56cfab2f9ada) but the input model was read into VRAM by [unsloth](https://github.com/unslothai/unsloth) to allow using the full 40k dataset to run on a single 3090.
57
 
58
  ## Benchmark (100.0*raw scores only)
59
 
 
62
  | Model | Average | IFEval | BHH | Math Lv5 | GPQA | MUSR | MMLU-PRO |
63
  | ----- | ------- | ------ | ----|--------- | ---- | ---- | -------- |
64
  | [gemma-2-2b-jpn-it](https://huggingface.co/datasets/open-llm-leaderboard/results/blob/main/google/gemma-2-2b-jpn-it/results_2024-10-15T15-21-39.173019.json) | 30.82 | 54.11 | 41.43 | 0.0 | 27.52 | 37.17 | 24.67 |
65
+ | [gemma-2-2b-jpn-it-abliterated-17-ORPO (4 epoches)](https://huggingface.co/datasets/open-llm-leaderboard/results/raw/main/ymcki/gemma-2-2b-jpn-it-abliterated-17-ORPO/results_2024-10-20T02-46-59.069357.json) | 29.99 | 50.94 | 38.59 | 2.87 | 27.43 | 38.23 | 21.86 |
66
+ | gemma-2-2b-jpn-it-abliterated-17-ORPO (8 epoches) | TBD | TBD | TBD | TBD | TBD | TBD | TBD |
67
+ | [gemma-2-2b-jpn-it-abliterated-18-ORPO (4 epoches)](https://huggingface.co/datasets/open-llm-leaderboard/results/raw/main/ymcki/gemma-2-2b-jpn-it-abliterated-18-ORPO/results_2024-10-22T04-04-56.385050.json) | 29.94 | 48.97 | 40.18 | 3.02 | 26.17 | 39.42 | 21.85 |
68
  | [gemma-2-2b-jpn-it-abliterated-17](https://huggingface.co/datasets/open-llm-leaderboard/results/raw/main/ymcki/gemma-2-2b-jpn-it-abliterated-17/results_2024-10-18T15-18-46.821674.json) | 30.29 | 52.65 | 40.46 | 0.0 | 27.18 | 36.90 | 24.55 |
69
  | [gemma-2-2b-jpn-it-abliterated-18](https://huggingface.co/datasets/open-llm-leaderboard/results/raw/main/ymcki/gemma-2-2b-jpn-it-abliterated-18/results_2024-10-18T15-41-42.399571.json) | 30.61 | 53.02 | 40.96 | 0.0 | 27.35 | 37.30 | 25.05 |
70
 
71
+ Looks like fine tuning is probably not enough. May need to run more epoches.
72
 
73
  ## How to run this model
74
 
 
77
  import transformers
78
  import torch
79
 
80
+ model_id = "gemma-2-2b-jpn-it-abliterated-17-ORPO"
81
  dtype = torch.bfloat16
82
 
83
  tokenizer = AutoTokenizer.from_pretrained(model_id)
 
103
  Then, you can target the specific file you want:
104
 
105
  ```
106
+ huggingface-cli download ymcki/gemma-2-2b-jpn-it-abliterated-17-ORPO --include "*" --local-dir ./
107
  ```
108
 
109
  ## Credits
110
 
111
+ Thank you mlabonne for describing his fine tuning method.
112
+
113
+ Thanks FullOf_Bad_Ideas from LocalLlama for the suggestion of using unsloth to save VRAM.
added_tokens.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "<|im_end|>": 256001,
3
+ "<|im_start|>": 256000
4
+ }
config.json CHANGED
@@ -1,15 +1,15 @@
1
  {
2
- "_name_or_path": "google/gemma-2-2b-it",
3
  "architectures": [
4
  "Gemma2ForCausalLM"
5
  ],
6
  "attention_bias": false,
7
  "attention_dropout": 0.0,
8
  "attn_logit_softcapping": 50.0,
9
- "bos_token_id": 2,
10
  "cache_implementation": "hybrid",
11
  "dtype": "bfloat16",
12
- "eos_token_id": 1,
13
  "final_logit_softcapping": 30.0,
14
  "head_dim": 256,
15
  "hidden_activation": "gelu_pytorch_tanh",
@@ -21,7 +21,7 @@
21
  "num_attention_heads": 8,
22
  "num_hidden_layers": 26,
23
  "num_key_value_heads": 4,
24
- "pad_token_id": 0,
25
  "query_pre_attn_scalar": 224,
26
  "rms_norm_eps": 1e-06,
27
  "rope_theta": 10000.0,
@@ -29,5 +29,5 @@
29
  "torch_dtype": "bfloat16",
30
  "transformers_version": "4.45.2",
31
  "use_cache": true,
32
- "vocab_size": 256000
33
  }
 
1
  {
2
+ "_name_or_path": "gemma-2-2b-jpn-it-abliterated-17",
3
  "architectures": [
4
  "Gemma2ForCausalLM"
5
  ],
6
  "attention_bias": false,
7
  "attention_dropout": 0.0,
8
  "attn_logit_softcapping": 50.0,
9
+ "bos_token_id": 256000,
10
  "cache_implementation": "hybrid",
11
  "dtype": "bfloat16",
12
+ "eos_token_id": 256001,
13
  "final_logit_softcapping": 30.0,
14
  "head_dim": 256,
15
  "hidden_activation": "gelu_pytorch_tanh",
 
21
  "num_attention_heads": 8,
22
  "num_hidden_layers": 26,
23
  "num_key_value_heads": 4,
24
+ "pad_token_id": 256001,
25
  "query_pre_attn_scalar": 224,
26
  "rms_norm_eps": 1e-06,
27
  "rope_theta": 10000.0,
 
29
  "torch_dtype": "bfloat16",
30
  "transformers_version": "4.45.2",
31
  "use_cache": true,
32
+ "vocab_size": 256002
33
  }
generation_config.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "_from_model_config": true,
3
- "bos_token_id": 2,
4
  "cache_implementation": "hybrid",
5
- "eos_token_id": 1,
6
- "pad_token_id": 0,
7
  "transformers_version": "4.45.2"
8
  }
 
1
  {
2
  "_from_model_config": true,
3
+ "bos_token_id": 256000,
4
  "cache_implementation": "hybrid",
5
+ "eos_token_id": 256001,
6
+ "pad_token_id": 256001,
7
  "transformers_version": "4.45.2"
8
  }
model.safetensors.index.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "metadata": {
3
- "total_size": 5228683776
4
  },
5
  "weight_map": {
6
  "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
 
1
  {
2
  "metadata": {
3
+ "total_size": 5228692992
4
  },
5
  "weight_map": {
6
  "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
special_tokens_map.json CHANGED
@@ -1,19 +1,23 @@
1
  {
2
- "bos_token": {
3
- "content": "<bos>",
4
- "lstrip": false,
5
- "normalized": false,
6
- "rstrip": false,
7
- "single_word": false
8
- },
9
- "eos_token": {
10
- "content": "<eos>",
11
- "lstrip": false,
12
- "normalized": false,
13
- "rstrip": false,
14
- "single_word": false
15
- },
16
- "pad_token": "<eos>",
 
 
 
 
17
  "unk_token": {
18
  "content": "<unk>",
19
  "lstrip": false,
 
1
  {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "<|im_start|>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "<|im_end|>",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ }
17
+ ],
18
+ "bos_token": "<|im_start|>",
19
+ "eos_token": "<|im_end|>",
20
+ "pad_token": "<|im_end|>",
21
  "unk_token": {
22
  "content": "<unk>",
23
  "lstrip": false,
tokenizer.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a5d1608a9eb188afeeb8a532bace0a4bcdc70afd57041b8441ee1940e3b19f72
3
- size 34363039
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:798ce05be77ad0639365c369e448442dd9a31f614551b88413d483c4d45c1839
3
+ size 34363415
tokenizer_config.json CHANGED
@@ -1993,14 +1993,38 @@
1993
  "rstrip": false,
1994
  "single_word": false,
1995
  "special": false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1996
  }
1997
  },
1998
- "bos_token": "<bos>",
1999
- "chat_template": "{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}",
 
 
 
 
2000
  "clean_up_tokenization_spaces": false,
2001
- "eos_token": "<eos>",
 
2002
  "model_max_length": 1000000000000000019884624838656,
2003
- "pad_token": "<eos>",
 
 
 
2004
  "sp_model_kwargs": {},
2005
  "spaces_between_special_tokens": false,
2006
  "tokenizer_class": "GemmaTokenizer",
 
1993
  "rstrip": false,
1994
  "single_word": false,
1995
  "special": false
1996
+ },
1997
+ "256000": {
1998
+ "content": "<|im_start|>",
1999
+ "lstrip": false,
2000
+ "normalized": false,
2001
+ "rstrip": false,
2002
+ "single_word": false,
2003
+ "special": true
2004
+ },
2005
+ "256001": {
2006
+ "content": "<|im_end|>",
2007
+ "lstrip": false,
2008
+ "normalized": false,
2009
+ "rstrip": false,
2010
+ "single_word": false,
2011
+ "special": true
2012
  }
2013
  },
2014
+ "additional_special_tokens": [
2015
+ "<|im_start|>",
2016
+ "<|im_end|>"
2017
+ ],
2018
+ "bos_token": "<|im_start|>",
2019
+ "chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
2020
  "clean_up_tokenization_spaces": false,
2021
+ "eos_token": "<|im_end|>",
2022
+ "max_length": null,
2023
  "model_max_length": 1000000000000000019884624838656,
2024
+ "pad_to_multiple_of": null,
2025
+ "pad_token": "<|im_end|>",
2026
+ "pad_token_type_id": 0,
2027
+ "padding_side": "left",
2028
  "sp_model_kwargs": {},
2029
  "spaces_between_special_tokens": false,
2030
  "tokenizer_class": "GemmaTokenizer",