Hugging Face's logo

LoneStriker
/
opus-v1.2-70b-2.4bpw-h6-exl2

Text Generation
Transformers
PyTorch
English
llama
unsloth
axolotl
text-generation-inference
Inference Endpoints
exl2
Model card Files Community
1
LoneStriker commited on
Commit
1044c8d
•
1 Parent(s): a655a2b

Upload folder using huggingface_hub

Browse files
Files changed (12) hide show
  1. README.md +238 -0
  2. added_tokens.json +4 -0
  3. config.json +39 -0
  4. generation_config.json +10 -0
  5. output-00001-of-00003.safetensors +3 -0
  6. output-00002-of-00003.safetensors +3 -0
  7. output-00003-of-00003.safetensors +3 -0
  8. pytorch_model.bin.index.json +730 -0
  9. special_tokens_map.json +40 -0
  10. tokenizer.json +0 -0
  11. tokenizer.model +3 -0
  12. tokenizer_config.json +64 -0
README.md ADDED
@@ -0,0 +1,238 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - unsloth
7
+ - axolotl
8
+ ---
9
+
10
+ # DreamGen Opus V1
11
+
12
+ <div style="display: flex; flex-direction: row; align-items: center;">
13
+ <img src="/dreamgen/opus-v1.2-70b/resolve/main/images/logo-1024.png" alt="model logo" style="
14
+ border-radius: 12px;
15
+ margin-right: 12px;
16
+ margin-top: 0px;
17
+ margin-bottom: 0px;
18
+ max-width: 100px;
19
+ height: auto;
20
+ "/>
21
+
22
+ Models for **(steerable) story-writing and role-playing**.
23
+ <br/>[All Opus V1 models, including quants](https://huggingface.co/collections/dreamgen/opus-v1-65d092a6f8ab7fc669111b31).
24
+
25
+ </div>
26
+
27
+ ## Resources
28
+
29
+ - [**Opus V1 prompting guide**](https://dreamgen.com/docs/models/opus/v1) with many (interactive) examples and prompts that you can copy.
30
+ - [**Google Colab**](https://colab.research.google.com/drive/1J178fH6IdQOXNi-Njgdacf5QgAxsdT20?usp=sharing) for interactive role-play using `opus-v1.2-7b`.
31
+ - [Python code](example/prompt/format.py) to format the prompt correctly.
32
+ - Join the community on [**Discord**](https://dreamgen.com/discord) to get early access to new models.
33
+
34
+ <img src="/dreamgen/opus-v1.2-70b/resolve/main/images/story_writing.webp" alt="story writing on dreamgen.com" style="
35
+ padding: 12px;
36
+ border-radius: 12px;
37
+ border: 2px solid #f9a8d4;
38
+ background: rgb(9, 9, 11);
39
+ "/>
40
+
41
+ ## Prompting
42
+
43
+ <details>
44
+ <summary>The models use an extended version of ChatML.</summary>
45
+
46
+ ```
47
+ <|im_start|>system
48
+ (Story description in the right format here)
49
+ (Typically consists of plot description, style description and characters)<|im_end|>
50
+ <|im_start|>user
51
+ (Your instruction on how the story should continue)<|im_end|>
52
+ <|im_start|>text names= Alice
53
+ (Continuation of the story from the Alice character)<|im_end|>
54
+ <|im_start|>text
55
+ (Continuation of the story from no character in particular (pure narration))<|im_end|>
56
+ <|im_start|>user
57
+ (Your instruction on how the story should continue)<|im_end|>
58
+ <|im_start|>text names= Bob
59
+ (Continuation of the story from the Bob character)<|im_end|>
60
+ ```
61
+
62
+ The Opus V1 extension is the addition of the `text` role, and the addition / modification of role names.
63
+
64
+ Pay attention to the following:
65
+
66
+ - The `text` messages can (but don't have to have) `names`, names are used to indicate the "active" character during role-play.
67
+ - There can be multiple subsequent message with a `text` role, especially if names are involved.
68
+ - There can be multiple names attached to a message.
69
+ - The format for names is `names= {{name[0]}}; {{name[1]}}`, beware of the spaces after `names=` and after the `;`. This spacing leads to most natural tokenization for the names.
70
+ </details>
71
+
72
+ While the main goal for the models is great story-writing and role-playing performance, the models are also capable of several writing related tasks as well as general assistance.
73
+
74
+ Here's how you can prompt the model for the following tasks
75
+
76
+ ### Steerable Story-writing and Role-playing:
77
+
78
+ - [Story-writing prompt guide](https://dreamgen.com/docs/models/opus/v1#task-story-writing)
79
+ - [Role-playing prompt guide](https://dreamgen.com/docs/models/opus/v1#task-role-playing)
80
+ - Input:
81
+ - System prompt: You provide story / role-play description, which consists of:
82
+ - Plot description
83
+ - Style description
84
+ - Characters and their descriptions
85
+ - Conversation turns:
86
+ - Text / message turn: This represents part of the story or role play
87
+ - Instruction: This tells the model what should happen next
88
+ - Output: Continuation of the story / role-play.
89
+
90
+ ### Story plot summarization
91
+
92
+ - [Prompt guide](https://dreamgen.com/docs/models/opus/v1#task-plot-description)
93
+ - Input: A story, or a few chapters of a story.
94
+ - Output: A description of the story or chapters.
95
+
96
+ #### Examples:
97
+
98
+ <details>
99
+ <summary>Summary of <a href="https://www.libraryofshortstories.com/onlinereader/the-last-incantation">short fiction story</a> in 50 words</summary>
100
+ <blockquote>
101
+ Malygris, a powerful magician, summons his long-dead lover Nylissa from the grave using necromancy. However, upon seeing her, he realizes that she is not the same person he once knew, leading him to question the nature of love, memory, and the power of magic.
102
+ </blockquote>
103
+ </details>
104
+ <details>
105
+ <summary>Summary of <a href="https://www.libraryofshortstories.com/onlinereader/the-last-incantation">short fiction story</a> in 200 words</summary>
106
+ <blockquote>
107
+ The story is a dark fantasy set in the city of Susran, where an ancient and powerful magician named Malygris dwells in a tower atop a conical hill. Malygris, who has mastered all forms of sorcery and necromancy, is plagued by a deep sense of ennui and melancholy. He seeks solace in the memory of Nylissa, a girl he once loved in his youth.
108
+
109
+ One day, Malygris decides to summon Nylissa's spirit from the grave using his necromantic powers. He performs the necessary rituals and incantations, and Nylissa appears before him as a ghostly apparition. However, as Malygris gazes upon her, he becomes increasingly disillusioned. He notices subtle changes in her appearance and behavior that do not match his memories of her. He begins to question whether this is truly the same Nylissa he once knew, or if it is just another illusion conjured by his magic.
110
+
111
+ As Malygris grapples with this uncertainty, he becomes increasingly despondent and disillusioned with life itself. He realizes that even his magic cannot bring back the past or make him truly happy. In the end, he dismisses Nylissa's spirit and returns to his lonely tower, resigned to his fate as an old and weary sorcerer.
112
+ </blockquote>
113
+
114
+ </details>
115
+ <details>
116
+ <summary>Summary of <a href="https://edition.cnn.com/2024/02/29/health/pets-diseases-viruses-bacteria-wellness/index.html">CNN News article</a> in 150 words</summary>
117
+ <blockquote>
118
+ The article discusses the recent human deaths from diseases transmitted by animals, specifically Alaskapox and bubonic plague. Dr. Leana Wen, an emergency physician and former Baltimore health commissioner, explains the various ways household pets can transmit diseases to humans. These include fecal-oral transmission, scratches and bites, and vector-borne diseases like ticks, fleas, and mosquitoes. Wen emphasizes the importance of vaccinating pets, maintaining cleanliness, and taking commonsense precautions to reduce disease transmission. She also advises that individuals who are immunocompromised, babies, and pregnant women should take additional care when interacting with animals.
119
+ </blockquote>
120
+ </details>
121
+ <details>
122
+ <summary>Summary of <a href="https://edition.cnn.com/2024/02/29/health/pets-diseases-viruses-bacteria-wellness/index.html">CNN News article</a> in 50 words</summary>
123
+ <blockquote>
124
+ A doctor discusses the risks of diseases transmitted from animals to humans, focusing on household pets like cats and dogs. They emphasize the importance of vaccination, hygiene, and regular veterinary care to prevent these illnesses.
125
+ </blockquote>
126
+ </details>
127
+
128
+ ### Story character description
129
+
130
+ - [Prompt guide](https://dreamgen.com/docs/models/opus/v1#task-char-description)
131
+ - Input: A story, or a few chapters of a story, set of characters.
132
+ - Output: A description of the characters.
133
+
134
+ #### Examples:
135
+
136
+ <details>
137
+ <summary>Description of the Malygris character from <a href="https://www.libraryofshortstories.com/onlinereader/the-last-incantation">this story</a> in 50 words</summary>
138
+ <blockquote>
139
+ Malygris is an ancient and powerful magician, characterized by his long white beard, cold green eyes, and a violet robe. He is a master of necromancy and possesses a vast knowledge of forbidden magic. Despite his immense power, he is plagued by a deep sense of melancholy and ennui.
140
+ </blockquote>
141
+ </details>
142
+
143
+ <details>
144
+ <summary>Description of the prince character from <a href="https://www.libraryofshortstories.com/onlinereader/the-pearl-of-love">this story</a> in 50 words</summary>
145
+ <blockquote>
146
+ The prince is a devoted and determined individual, driven by his love for his late queen. He is described as a graceful youth in the beginning, but as he grows older, his dedication to the Pearl of Love transforms him into a grave and intent man. His sense of color and design evolves over time, reflecting his growing maturity and artistic vision.
147
+ </blockquote>
148
+ </details>
149
+
150
+ ### Story style description
151
+
152
+ - [Prompt guide](https://dreamgen.com/docs/models/opus/v1#task-style-description)
153
+ - Input: A story, or a few chapters of a story.
154
+ - Output: A description the style of the story.
155
+
156
+ <details>
157
+ <summary>Style description of <a href="https://www.libraryofshortstories.com/onlinereader/the-last-incantation">this story</a></summary>
158
+ <blockquote>
159
+ The story is written in a descriptive and evocative style, with a focus on the atmosphere and setting. The author uses vivid and detailed language to create a sense of place and time, and employs a variety of literary devices such as similes, metaphors, and allusions to enhance the mood and tone of the narrative. The sentence structure is varied, with a mix of short and long sentences that create a rhythmic flow to the prose.
160
+ </blockquote>
161
+ </details>
162
+
163
+ <details>
164
+ <summary>Style description of <a href="https://www.libraryofshortstories.com/onlinereader/the-pearl-of-love">this story</a></summary>
165
+ <blockquote>
166
+ The writing style is descriptive and evocative, with a focus on the beauty and grandeur of the Pearl of Love. The author uses vivid imagery and sensory details to create a rich and immersive atmosphere. The tone is reverential and contemplative, reflecting the prince's deep love for his queen and his dedication to creating a lasting monument to her memory.
167
+ </blockquote>
168
+ </details>
169
+
170
+ ### Story description to chapters
171
+
172
+ - [Prompt guide](https://dreamgen.com/docs/models/opus/v1#task-story-description-to-chapter-descriptions)
173
+ - Input: A brief plot description and the desired number of chapters.
174
+ - Output: A description for each chapter.
175
+
176
+ ### And more...
177
+
178
+ ## Sampling params
179
+
180
+ For story-writing and role-play, I recommend "Min P" based sampling with `min_p` in the range `[0.01, 0.1]` and with `temperature` in the range `[0.5, 1.5]`, depending on your preferences. A good starting point would be `min_p=0.1; temperature=0.8`.
181
+
182
+ You may also benefit from setting presence, frequency and repetition penalties, especially at lower temperatures.
183
+
184
+ ## Dataset
185
+
186
+ The fine-tuning dataset consisted of ~100M tokens of steerable story-writing, role-playing, writing-assistant and general-assistant examples. Each example was up to 31000 tokens long.
187
+
188
+ All story-writing and role-playing examples were based on human-written text.
189
+
190
+ ![token count distribution](images/token_count_cum__token_bucket.png)
191
+
192
+ ## Running the model
193
+
194
+ The model is should be compatible with any software that supports the base model, but beware of prompting and tokenization.
195
+
196
+ I recommend using these model versions:
197
+
198
+ - 7B: [no quant (opus-v1.2-7b)](https://huggingface.co/dreamgen/opus-v1.2-7b)
199
+ - 34B: [no quant (opus-v1-34b)](https://huggingface.co/dreamgen/opus-v1-34b) or [awq (opus-v1-34b-awq)](https://huggingface.co/dreamgen/opus-v1-34b-awq)
200
+ - 34B: [no quant (opus-v1.2-70b)](https://huggingface.co/dreamgen/opus-v1.2-70b) or [awq (opus-v1.2-70b-awq)](https://huggingface.co/dreamgen/opus-v1.2-70b-awq)
201
+
202
+ ### Running on DreamGen.com (free)
203
+
204
+ You can run the models on [dreamgen.com](https://dreamgen.com) for free — you can use the built-in UI for story-writing & role-playing, or use [the API](https://dreamgen.com/docs/api).
205
+
206
+ ### Running Locally
207
+
208
+ - **Make sure your prompt is as close as possible to the Opus V1**
209
+ - Regardless of which backend you use, it's important that you format your prompt well and that the tokenization works correctly.
210
+ - [Read the prompt guide](https://dreamgen.com/docs/models/opus/v1)
211
+ - [Read the prompt formatting code](example/prompt/format.py)
212
+ - Make sure `<|im_start|>` and `<|im_end|>` are tokenized correctly
213
+ - **vLLM**
214
+ - [**Google Colab**](https://colab.research.google.com/drive/1J178fH6IdQOXNi-Njgdacf5QgAxsdT20?usp=sharing): This is a simple interactive Google Colab to do role-play with the 7B model, it should fit on the T4 GPU.
215
+ - [Code](example/prompt/interactive.py): This is simple script for interactive chat for one hard-coded scenario.
216
+ - **SillyTavern**
217
+ - [Settings](https://huggingface.co/dreamgen/opus-v1.2-70b/tree/main/configs/silly_tavern), v2 kindly provided by @MarinaraSpaghetti
218
+ - [Settings screenshot](configs/silly_tavern/settings_screenshot.webp)
219
+ - Make sure to unselect `Skip special tokens`, otherwise it won't work
220
+ - This is just an attempt at approximating the Opus V1 prompt, it won't be perfect
221
+ - **LM Studio**
222
+ - [Config](configs/lmstudio/preset.json)
223
+ - Just like ChatML, just changed "assistant" to "text" role.
224
+ - **There's a bug** in LM Studio if you delete a message or click "Continue", [see here for details](https://discord.com/channels/1110598183144399058/1212665261128417280/1212665261128417280).
225
+ - **HuggingFace**
226
+ - [Chat template](tokenizer_config.json#L51)
227
+ - Just like ChatML, just changed "assistant" to "text" role.
228
+
229
+ ## Known Issues
230
+
231
+ - **34B repetition**:
232
+ - The 34B sometimes gets stuck repeating the same word, or synonyms. This seems to be a common problem across various Yi 34B fine-tunes.
233
+ - **GGUF**:
234
+ - The tokenization might be messed up. Some users reported that `<|im_start|>` and `<|im_end|>` are tokenized as multiple tokens. Also llama.cpp may not tokenize correctly (the Yi tokenizer is subtly different from the Llama 2 tokenizer).
235
+
236
+ ## License
237
+
238
+ - This model is intended for personal use only, other use is not permitted.
added_tokens.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "<|im_end|>": 32001,
3
+ "<|im_start|>": 32000
4
+ }
config.json ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "meta-llama/Llama-2-70b-hf",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 1,
9
+ "eos_token_id": 2,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 8192,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 32768,
15
+ "model_type": "llama",
16
+ "num_attention_heads": 64,
17
+ "num_hidden_layers": 80,
18
+ "num_key_value_heads": 8,
19
+ "pretraining_tp": 1,
20
+ "rms_norm_eps": 1e-05,
21
+ "rope_scaling": null,
22
+ "rope_theta": 1000000,
23
+ "tie_word_embeddings": false,
24
+ "torch_dtype": "bfloat16",
25
+ "transformers_version": "4.39.0.dev0",
26
+ "use_cache": false,
27
+ "vocab_size": 32002,
28
+ "quantization_config": {
29
+ "quant_method": "exl2",
30
+ "version": "0.0.15",
31
+ "bits": 2.4,
32
+ "head_bits": 6,
33
+ "calibration": {
34
+ "rows": 100,
35
+ "length": 2048,
36
+ "dataset": "(default)"
37
+ }
38
+ }
39
+ }
generation_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 1,
3
+ "do_sample": true,
4
+ "eos_token_id": 2,
5
+ "max_length": 4096,
6
+ "pad_token_id": 0,
7
+ "temperature": 0.6,
8
+ "top_p": 0.9,
9
+ "transformers_version": "4.39.0.dev0"
10
+ }
output-00001-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0991e91dc6167a1befe964c5c78c3f5ecfaaedb7f20719ced52e2ef53482ef15
3
+ size 8566989058
output-00002-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad59ff3249866b3fd1b02906377f0e78b77bf81ddac21b396fb56459e9af4924
3
+ size 8586494940
output-00003-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60472522c94d9b5e13ff9b12b5d6ef12250ccb94f76b581732349c8ae3a2e8c2
3
+ size 4115586084
pytorch_model.bin.index.json ADDED
@@ -0,0 +1,730 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 137953361920
4
+ },
5
+ "weight_map": {
6
+ "lm_head.weight": "pytorch_model-00029-of-00029.bin",
7
+ "model.embed_tokens.weight": "pytorch_model-00001-of-00029.bin",
8
+ "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00029.bin",
9
+ "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00029.bin",
10
+ "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00029.bin",
11
+ "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00029.bin",
12
+ "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00029.bin",
13
+ "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00029.bin",
14
+ "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00029.bin",
15
+ "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00029.bin",
16
+ "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00029.bin",
17
+ "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00029.bin",
18
+ "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00029.bin",
19
+ "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00029.bin",
20
+ "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00029.bin",
21
+ "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00029.bin",
22
+ "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00029.bin",
23
+ "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00029.bin",
24
+ "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00029.bin",
25
+ "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00029.bin",
26
+ "model.layers.10.input_layernorm.weight": "pytorch_model-00004-of-00029.bin",
27
+ "model.layers.10.mlp.down_proj.weight": "pytorch_model-00004-of-00029.bin",
28
+ "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00004-of-00029.bin",
29
+ "model.layers.10.mlp.up_proj.weight": "pytorch_model-00004-of-00029.bin",
30
+ "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00004-of-00029.bin",
31
+ "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00004-of-00029.bin",
32
+ "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00004-of-00029.bin",
33
+ "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00004-of-00029.bin",
34
+ "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00004-of-00029.bin",
35
+ "model.layers.11.input_layernorm.weight": "pytorch_model-00005-of-00029.bin",
36
+ "model.layers.11.mlp.down_proj.weight": "pytorch_model-00005-of-00029.bin",
37
+ "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00005-of-00029.bin",
38
+ "model.layers.11.mlp.up_proj.weight": "pytorch_model-00005-of-00029.bin",
39
+ "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00005-of-00029.bin",
40
+ "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00005-of-00029.bin",
41
+ "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00005-of-00029.bin",
42
+ "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00005-of-00029.bin",
43
+ "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00005-of-00029.bin",
44
+ "model.layers.12.input_layernorm.weight": "pytorch_model-00005-of-00029.bin",
45
+ "model.layers.12.mlp.down_proj.weight": "pytorch_model-00005-of-00029.bin",
46
+ "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00005-of-00029.bin",
47
+ "model.layers.12.mlp.up_proj.weight": "pytorch_model-00005-of-00029.bin",
48
+ "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00005-of-00029.bin",
49
+ "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00005-of-00029.bin",
50
+ "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00005-of-00029.bin",
51
+ "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00005-of-00029.bin",
52
+ "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00005-of-00029.bin",
53
+ "model.layers.13.input_layernorm.weight": "pytorch_model-00006-of-00029.bin",
54
+ "model.layers.13.mlp.down_proj.weight": "pytorch_model-00006-of-00029.bin",
55
+ "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00005-of-00029.bin",
56
+ "model.layers.13.mlp.up_proj.weight": "pytorch_model-00005-of-00029.bin",
57
+ "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00006-of-00029.bin",
58
+ "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00005-of-00029.bin",
59
+ "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00005-of-00029.bin",
60
+ "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00005-of-00029.bin",
61
+ "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00005-of-00029.bin",
62
+ "model.layers.14.input_layernorm.weight": "pytorch_model-00006-of-00029.bin",
63
+ "model.layers.14.mlp.down_proj.weight": "pytorch_model-00006-of-00029.bin",
64
+ "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00006-of-00029.bin",
65
+ "model.layers.14.mlp.up_proj.weight": "pytorch_model-00006-of-00029.bin",
66
+ "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00006-of-00029.bin",
67
+ "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00006-of-00029.bin",
68
+ "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00006-of-00029.bin",
69
+ "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00006-of-00029.bin",
70
+ "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00006-of-00029.bin",
71
+ "model.layers.15.input_layernorm.weight": "pytorch_model-00006-of-00029.bin",
72
+ "model.layers.15.mlp.down_proj.weight": "pytorch_model-00006-of-00029.bin",
73
+ "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00006-of-00029.bin",
74
+ "model.layers.15.mlp.up_proj.weight": "pytorch_model-00006-of-00029.bin",
75
+ "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00006-of-00029.bin",
76
+ "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00006-of-00029.bin",
77
+ "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00006-of-00029.bin",
78
+ "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00006-of-00029.bin",
79
+ "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00006-of-00029.bin",
80
+ "model.layers.16.input_layernorm.weight": "pytorch_model-00007-of-00029.bin",
81
+ "model.layers.16.mlp.down_proj.weight": "pytorch_model-00007-of-00029.bin",
82
+ "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00006-of-00029.bin",
83
+ "model.layers.16.mlp.up_proj.weight": "pytorch_model-00007-of-00029.bin",
84
+ "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00007-of-00029.bin",
85
+ "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00006-of-00029.bin",
86
+ "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00006-of-00029.bin",
87
+ "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00006-of-00029.bin",
88
+ "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00006-of-00029.bin",
89
+ "model.layers.17.input_layernorm.weight": "pytorch_model-00007-of-00029.bin",
90
+ "model.layers.17.mlp.down_proj.weight": "pytorch_model-00007-of-00029.bin",
91
+ "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00007-of-00029.bin",
92
+ "model.layers.17.mlp.up_proj.weight": "pytorch_model-00007-of-00029.bin",
93
+ "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00007-of-00029.bin",
94
+ "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00007-of-00029.bin",
95
+ "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00007-of-00029.bin",
96
+ "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00007-of-00029.bin",
97
+ "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00007-of-00029.bin",
98
+ "model.layers.18.input_layernorm.weight": "pytorch_model-00007-of-00029.bin",
99
+ "model.layers.18.mlp.down_proj.weight": "pytorch_model-00007-of-00029.bin",
100
+ "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00007-of-00029.bin",
101
+ "model.layers.18.mlp.up_proj.weight": "pytorch_model-00007-of-00029.bin",
102
+ "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00007-of-00029.bin",
103
+ "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00007-of-00029.bin",
104
+ "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00007-of-00029.bin",
105
+ "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00007-of-00029.bin",
106
+ "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00007-of-00029.bin",
107
+ "model.layers.19.input_layernorm.weight": "pytorch_model-00008-of-00029.bin",
108
+ "model.layers.19.mlp.down_proj.weight": "pytorch_model-00008-of-00029.bin",
109
+ "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00008-of-00029.bin",
110
+ "model.layers.19.mlp.up_proj.weight": "pytorch_model-00008-of-00029.bin",
111
+ "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00008-of-00029.bin",
112
+ "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00007-of-00029.bin",
113
+ "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00007-of-00029.bin",
114
+ "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00007-of-00029.bin",
115
+ "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00007-of-00029.bin",
116
+ "model.layers.2.input_layernorm.weight": "pytorch_model-00002-of-00029.bin",
117
+ "model.layers.2.mlp.down_proj.weight": "pytorch_model-00002-of-00029.bin",
118
+ "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00029.bin",
119
+ "model.layers.2.mlp.up_proj.weight": "pytorch_model-00002-of-00029.bin",
120
+ "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00002-of-00029.bin",
121
+ "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00029.bin",
122
+ "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00029.bin",
123
+ "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00029.bin",
124
+ "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00029.bin",
125
+ "model.layers.20.input_layernorm.weight": "pytorch_model-00008-of-00029.bin",
126
+ "model.layers.20.mlp.down_proj.weight": "pytorch_model-00008-of-00029.bin",
127
+ "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00008-of-00029.bin",
128
+ "model.layers.20.mlp.up_proj.weight": "pytorch_model-00008-of-00029.bin",
129
+ "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00008-of-00029.bin",
130
+ "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00008-of-00029.bin",
131
+ "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00008-of-00029.bin",
132
+ "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00008-of-00029.bin",
133
+ "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00008-of-00029.bin",
134
+ "model.layers.21.input_layernorm.weight": "pytorch_model-00008-of-00029.bin",
135
+ "model.layers.21.mlp.down_proj.weight": "pytorch_model-00008-of-00029.bin",
136
+ "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00008-of-00029.bin",
137
+ "model.layers.21.mlp.up_proj.weight": "pytorch_model-00008-of-00029.bin",
138
+ "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00008-of-00029.bin",
139
+ "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00008-of-00029.bin",
140
+ "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00008-of-00029.bin",
141
+ "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00008-of-00029.bin",
142
+ "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00008-of-00029.bin",
143
+ "model.layers.22.input_layernorm.weight": "pytorch_model-00009-of-00029.bin",
144
+ "model.layers.22.mlp.down_proj.weight": "pytorch_model-00009-of-00029.bin",
145
+ "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00009-of-00029.bin",
146
+ "model.layers.22.mlp.up_proj.weight": "pytorch_model-00009-of-00029.bin",
147
+ "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00009-of-00029.bin",
148
+ "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00008-of-00029.bin",
149
+ "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00009-of-00029.bin",
150
+ "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00008-of-00029.bin",
151
+ "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00008-of-00029.bin",
152
+ "model.layers.23.input_layernorm.weight": "pytorch_model-00009-of-00029.bin",
153
+ "model.layers.23.mlp.down_proj.weight": "pytorch_model-00009-of-00029.bin",
154
+ "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00009-of-00029.bin",
155
+ "model.layers.23.mlp.up_proj.weight": "pytorch_model-00009-of-00029.bin",
156
+ "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00009-of-00029.bin",
157
+ "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00009-of-00029.bin",
158
+ "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00009-of-00029.bin",
159
+ "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00009-of-00029.bin",
160
+ "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00009-of-00029.bin",
161
+ "model.layers.24.input_layernorm.weight": "pytorch_model-00009-of-00029.bin",
162
+ "model.layers.24.mlp.down_proj.weight": "pytorch_model-00009-of-00029.bin",
163
+ "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00009-of-00029.bin",
164
+ "model.layers.24.mlp.up_proj.weight": "pytorch_model-00009-of-00029.bin",
165
+ "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00009-of-00029.bin",
166
+ "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00009-of-00029.bin",
167
+ "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00009-of-00029.bin",
168
+ "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00009-of-00029.bin",
169
+ "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00009-of-00029.bin",
170
+ "model.layers.25.input_layernorm.weight": "pytorch_model-00010-of-00029.bin",
171
+ "model.layers.25.mlp.down_proj.weight": "pytorch_model-00010-of-00029.bin",
172
+ "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00010-of-00029.bin",
173
+ "model.layers.25.mlp.up_proj.weight": "pytorch_model-00010-of-00029.bin",
174
+ "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00010-of-00029.bin",
175
+ "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00010-of-00029.bin",
176
+ "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00010-of-00029.bin",
177
+ "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00010-of-00029.bin",
178
+ "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00010-of-00029.bin",
179
+ "model.layers.26.input_layernorm.weight": "pytorch_model-00010-of-00029.bin",
180
+ "model.layers.26.mlp.down_proj.weight": "pytorch_model-00010-of-00029.bin",
181
+ "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00010-of-00029.bin",
182
+ "model.layers.26.mlp.up_proj.weight": "pytorch_model-00010-of-00029.bin",
183
+ "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00010-of-00029.bin",
184
+ "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00010-of-00029.bin",
185
+ "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00010-of-00029.bin",
186
+ "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00010-of-00029.bin",
187
+ "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00010-of-00029.bin",
188
+ "model.layers.27.input_layernorm.weight": "pytorch_model-00011-of-00029.bin",
189
+ "model.layers.27.mlp.down_proj.weight": "pytorch_model-00011-of-00029.bin",
190
+ "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00010-of-00029.bin",
191
+ "model.layers.27.mlp.up_proj.weight": "pytorch_model-00010-of-00029.bin",
192
+ "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00011-of-00029.bin",
193
+ "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00010-of-00029.bin",
194
+ "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00010-of-00029.bin",
195
+ "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00010-of-00029.bin",
196
+ "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00010-of-00029.bin",
197
+ "model.layers.28.input_layernorm.weight": "pytorch_model-00011-of-00029.bin",
198
+ "model.layers.28.mlp.down_proj.weight": "pytorch_model-00011-of-00029.bin",
199
+ "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00011-of-00029.bin",
200
+ "model.layers.28.mlp.up_proj.weight": "pytorch_model-00011-of-00029.bin",
201
+ "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00011-of-00029.bin",
202
+ "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00011-of-00029.bin",
203
+ "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00011-of-00029.bin",
204
+ "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00011-of-00029.bin",
205
+ "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00011-of-00029.bin",
206
+ "model.layers.29.input_layernorm.weight": "pytorch_model-00011-of-00029.bin",
207
+ "model.layers.29.mlp.down_proj.weight": "pytorch_model-00011-of-00029.bin",
208
+ "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00011-of-00029.bin",
209
+ "model.layers.29.mlp.up_proj.weight": "pytorch_model-00011-of-00029.bin",
210
+ "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00011-of-00029.bin",
211
+ "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00011-of-00029.bin",
212
+ "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00011-of-00029.bin",
213
+ "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00011-of-00029.bin",
214
+ "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00011-of-00029.bin",
215
+ "model.layers.3.input_layernorm.weight": "pytorch_model-00002-of-00029.bin",
216
+ "model.layers.3.mlp.down_proj.weight": "pytorch_model-00002-of-00029.bin",
217
+ "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00002-of-00029.bin",
218
+ "model.layers.3.mlp.up_proj.weight": "pytorch_model-00002-of-00029.bin",
219
+ "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00002-of-00029.bin",
220
+ "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00002-of-00029.bin",
221
+ "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00002-of-00029.bin",
222
+ "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00002-of-00029.bin",
223
+ "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00002-of-00029.bin",
224
+ "model.layers.30.input_layernorm.weight": "pytorch_model-00012-of-00029.bin",
225
+ "model.layers.30.mlp.down_proj.weight": "pytorch_model-00012-of-00029.bin",
226
+ "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00011-of-00029.bin",
227
+ "model.layers.30.mlp.up_proj.weight": "pytorch_model-00012-of-00029.bin",
228
+ "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00012-of-00029.bin",
229
+ "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00011-of-00029.bin",
230
+ "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00011-of-00029.bin",
231
+ "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00011-of-00029.bin",
232
+ "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00011-of-00029.bin",
233
+ "model.layers.31.input_layernorm.weight": "pytorch_model-00012-of-00029.bin",
234
+ "model.layers.31.mlp.down_proj.weight": "pytorch_model-00012-of-00029.bin",
235
+ "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00012-of-00029.bin",
236
+ "model.layers.31.mlp.up_proj.weight": "pytorch_model-00012-of-00029.bin",
237
+ "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00012-of-00029.bin",
238
+ "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00012-of-00029.bin",
239
+ "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00012-of-00029.bin",
240
+ "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00012-of-00029.bin",
241
+ "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00012-of-00029.bin",
242
+ "model.layers.32.input_layernorm.weight": "pytorch_model-00012-of-00029.bin",
243
+ "model.layers.32.mlp.down_proj.weight": "pytorch_model-00012-of-00029.bin",
244
+ "model.layers.32.mlp.gate_proj.weight": "pytorch_model-00012-of-00029.bin",
245
+ "model.layers.32.mlp.up_proj.weight": "pytorch_model-00012-of-00029.bin",
246
+ "model.layers.32.post_attention_layernorm.weight": "pytorch_model-00012-of-00029.bin",
247
+ "model.layers.32.self_attn.k_proj.weight": "pytorch_model-00012-of-00029.bin",
248
+ "model.layers.32.self_attn.o_proj.weight": "pytorch_model-00012-of-00029.bin",
249
+ "model.layers.32.self_attn.q_proj.weight": "pytorch_model-00012-of-00029.bin",
250
+ "model.layers.32.self_attn.v_proj.weight": "pytorch_model-00012-of-00029.bin",
251
+ "model.layers.33.input_layernorm.weight": "pytorch_model-00013-of-00029.bin",
252
+ "model.layers.33.mlp.down_proj.weight": "pytorch_model-00013-of-00029.bin",
253
+ "model.layers.33.mlp.gate_proj.weight": "pytorch_model-00013-of-00029.bin",
254
+ "model.layers.33.mlp.up_proj.weight": "pytorch_model-00013-of-00029.bin",
255
+ "model.layers.33.post_attention_layernorm.weight": "pytorch_model-00013-of-00029.bin",
256
+ "model.layers.33.self_attn.k_proj.weight": "pytorch_model-00012-of-00029.bin",
257
+ "model.layers.33.self_attn.o_proj.weight": "pytorch_model-00012-of-00029.bin",
258
+ "model.layers.33.self_attn.q_proj.weight": "pytorch_model-00012-of-00029.bin",
259
+ "model.layers.33.self_attn.v_proj.weight": "pytorch_model-00012-of-00029.bin",
260
+ "model.layers.34.input_layernorm.weight": "pytorch_model-00013-of-00029.bin",
261
+ "model.layers.34.mlp.down_proj.weight": "pytorch_model-00013-of-00029.bin",
262
+ "model.layers.34.mlp.gate_proj.weight": "pytorch_model-00013-of-00029.bin",
263
+ "model.layers.34.mlp.up_proj.weight": "pytorch_model-00013-of-00029.bin",
264
+ "model.layers.34.post_attention_layernorm.weight": "pytorch_model-00013-of-00029.bin",
265
+ "model.layers.34.self_attn.k_proj.weight": "pytorch_model-00013-of-00029.bin",
266
+ "model.layers.34.self_attn.o_proj.weight": "pytorch_model-00013-of-00029.bin",
267
+ "model.layers.34.self_attn.q_proj.weight": "pytorch_model-00013-of-00029.bin",
268
+ "model.layers.34.self_attn.v_proj.weight": "pytorch_model-00013-of-00029.bin",
269
+ "model.layers.35.input_layernorm.weight": "pytorch_model-00013-of-00029.bin",
270
+ "model.layers.35.mlp.down_proj.weight": "pytorch_model-00013-of-00029.bin",
271
+ "model.layers.35.mlp.gate_proj.weight": "pytorch_model-00013-of-00029.bin",
272
+ "model.layers.35.mlp.up_proj.weight": "pytorch_model-00013-of-00029.bin",
273
+ "model.layers.35.post_attention_layernorm.weight": "pytorch_model-00013-of-00029.bin",
274
+ "model.layers.35.self_attn.k_proj.weight": "pytorch_model-00013-of-00029.bin",
275
+ "model.layers.35.self_attn.o_proj.weight": "pytorch_model-00013-of-00029.bin",
276
+ "model.layers.35.self_attn.q_proj.weight": "pytorch_model-00013-of-00029.bin",
277
+ "model.layers.35.self_attn.v_proj.weight": "pytorch_model-00013-of-00029.bin",
278
+ "model.layers.36.input_layernorm.weight": "pytorch_model-00014-of-00029.bin",
279
+ "model.layers.36.mlp.down_proj.weight": "pytorch_model-00014-of-00029.bin",
280
+ "model.layers.36.mlp.gate_proj.weight": "pytorch_model-00014-of-00029.bin",
281
+ "model.layers.36.mlp.up_proj.weight": "pytorch_model-00014-of-00029.bin",
282
+ "model.layers.36.post_attention_layernorm.weight": "pytorch_model-00014-of-00029.bin",
283
+ "model.layers.36.self_attn.k_proj.weight": "pytorch_model-00013-of-00029.bin",
284
+ "model.layers.36.self_attn.o_proj.weight": "pytorch_model-00014-of-00029.bin",
285
+ "model.layers.36.self_attn.q_proj.weight": "pytorch_model-00013-of-00029.bin",
286
+ "model.layers.36.self_attn.v_proj.weight": "pytorch_model-00013-of-00029.bin",
287
+ "model.layers.37.input_layernorm.weight": "pytorch_model-00014-of-00029.bin",
288
+ "model.layers.37.mlp.down_proj.weight": "pytorch_model-00014-of-00029.bin",
289
+ "model.layers.37.mlp.gate_proj.weight": "pytorch_model-00014-of-00029.bin",
290
+ "model.layers.37.mlp.up_proj.weight": "pytorch_model-00014-of-00029.bin",
291
+ "model.layers.37.post_attention_layernorm.weight": "pytorch_model-00014-of-00029.bin",
292
+ "model.layers.37.self_attn.k_proj.weight": "pytorch_model-00014-of-00029.bin",
293
+ "model.layers.37.self_attn.o_proj.weight": "pytorch_model-00014-of-00029.bin",
294
+ "model.layers.37.self_attn.q_proj.weight": "pytorch_model-00014-of-00029.bin",
295
+ "model.layers.37.self_attn.v_proj.weight": "pytorch_model-00014-of-00029.bin",
296
+ "model.layers.38.input_layernorm.weight": "pytorch_model-00014-of-00029.bin",
297
+ "model.layers.38.mlp.down_proj.weight": "pytorch_model-00014-of-00029.bin",
298
+ "model.layers.38.mlp.gate_proj.weight": "pytorch_model-00014-of-00029.bin",
299
+ "model.layers.38.mlp.up_proj.weight": "pytorch_model-00014-of-00029.bin",
300
+ "model.layers.38.post_attention_layernorm.weight": "pytorch_model-00014-of-00029.bin",
301
+ "model.layers.38.self_attn.k_proj.weight": "pytorch_model-00014-of-00029.bin",
302
+ "model.layers.38.self_attn.o_proj.weight": "pytorch_model-00014-of-00029.bin",
303
+ "model.layers.38.self_attn.q_proj.weight": "pytorch_model-00014-of-00029.bin",
304
+ "model.layers.38.self_attn.v_proj.weight": "pytorch_model-00014-of-00029.bin",
305
+ "model.layers.39.input_layernorm.weight": "pytorch_model-00015-of-00029.bin",
306
+ "model.layers.39.mlp.down_proj.weight": "pytorch_model-00015-of-00029.bin",
307
+ "model.layers.39.mlp.gate_proj.weight": "pytorch_model-00015-of-00029.bin",
308
+ "model.layers.39.mlp.up_proj.weight": "pytorch_model-00015-of-00029.bin",
309
+ "model.layers.39.post_attention_layernorm.weight": "pytorch_model-00015-of-00029.bin",
310
+ "model.layers.39.self_attn.k_proj.weight": "pytorch_model-00015-of-00029.bin",
311
+ "model.layers.39.self_attn.o_proj.weight": "pytorch_model-00015-of-00029.bin",
312
+ "model.layers.39.self_attn.q_proj.weight": "pytorch_model-00015-of-00029.bin",
313
+ "model.layers.39.self_attn.v_proj.weight": "pytorch_model-00015-of-00029.bin",
314
+ "model.layers.4.input_layernorm.weight": "pytorch_model-00002-of-00029.bin",
315
+ "model.layers.4.mlp.down_proj.weight": "pytorch_model-00002-of-00029.bin",
316
+ "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00002-of-00029.bin",
317
+ "model.layers.4.mlp.up_proj.weight": "pytorch_model-00002-of-00029.bin",
318
+ "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00002-of-00029.bin",
319
+ "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00002-of-00029.bin",
320
+ "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00002-of-00029.bin",
321
+ "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00002-of-00029.bin",
322
+ "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00002-of-00029.bin",
323
+ "model.layers.40.input_layernorm.weight": "pytorch_model-00015-of-00029.bin",
324
+ "model.layers.40.mlp.down_proj.weight": "pytorch_model-00015-of-00029.bin",
325
+ "model.layers.40.mlp.gate_proj.weight": "pytorch_model-00015-of-00029.bin",
326
+ "model.layers.40.mlp.up_proj.weight": "pytorch_model-00015-of-00029.bin",
327
+ "model.layers.40.post_attention_layernorm.weight": "pytorch_model-00015-of-00029.bin",
328
+ "model.layers.40.self_attn.k_proj.weight": "pytorch_model-00015-of-00029.bin",
329
+ "model.layers.40.self_attn.o_proj.weight": "pytorch_model-00015-of-00029.bin",
330
+ "model.layers.40.self_attn.q_proj.weight": "pytorch_model-00015-of-00029.bin",
331
+ "model.layers.40.self_attn.v_proj.weight": "pytorch_model-00015-of-00029.bin",
332
+ "model.layers.41.input_layernorm.weight": "pytorch_model-00016-of-00029.bin",
333
+ "model.layers.41.mlp.down_proj.weight": "pytorch_model-00016-of-00029.bin",
334
+ "model.layers.41.mlp.gate_proj.weight": "pytorch_model-00015-of-00029.bin",
335
+ "model.layers.41.mlp.up_proj.weight": "pytorch_model-00015-of-00029.bin",
336
+ "model.layers.41.post_attention_layernorm.weight": "pytorch_model-00016-of-00029.bin",
337
+ "model.layers.41.self_attn.k_proj.weight": "pytorch_model-00015-of-00029.bin",
338
+ "model.layers.41.self_attn.o_proj.weight": "pytorch_model-00015-of-00029.bin",
339
+ "model.layers.41.self_attn.q_proj.weight": "pytorch_model-00015-of-00029.bin",
340
+ "model.layers.41.self_attn.v_proj.weight": "pytorch_model-00015-of-00029.bin",
341
+ "model.layers.42.input_layernorm.weight": "pytorch_model-00016-of-00029.bin",
342
+ "model.layers.42.mlp.down_proj.weight": "pytorch_model-00016-of-00029.bin",
343
+ "model.layers.42.mlp.gate_proj.weight": "pytorch_model-00016-of-00029.bin",
344
+ "model.layers.42.mlp.up_proj.weight": "pytorch_model-00016-of-00029.bin",
345
+ "model.layers.42.post_attention_layernorm.weight": "pytorch_model-00016-of-00029.bin",
346
+ "model.layers.42.self_attn.k_proj.weight": "pytorch_model-00016-of-00029.bin",
347
+ "model.layers.42.self_attn.o_proj.weight": "pytorch_model-00016-of-00029.bin",
348
+ "model.layers.42.self_attn.q_proj.weight": "pytorch_model-00016-of-00029.bin",
349
+ "model.layers.42.self_attn.v_proj.weight": "pytorch_model-00016-of-00029.bin",
350
+ "model.layers.43.input_layernorm.weight": "pytorch_model-00016-of-00029.bin",
351
+ "model.layers.43.mlp.down_proj.weight": "pytorch_model-00016-of-00029.bin",
352
+ "model.layers.43.mlp.gate_proj.weight": "pytorch_model-00016-of-00029.bin",
353
+ "model.layers.43.mlp.up_proj.weight": "pytorch_model-00016-of-00029.bin",
354
+ "model.layers.43.post_attention_layernorm.weight": "pytorch_model-00016-of-00029.bin",
355
+ "model.layers.43.self_attn.k_proj.weight": "pytorch_model-00016-of-00029.bin",
356
+ "model.layers.43.self_attn.o_proj.weight": "pytorch_model-00016-of-00029.bin",
357
+ "model.layers.43.self_attn.q_proj.weight": "pytorch_model-00016-of-00029.bin",
358
+ "model.layers.43.self_attn.v_proj.weight": "pytorch_model-00016-of-00029.bin",
359
+ "model.layers.44.input_layernorm.weight": "pytorch_model-00017-of-00029.bin",
360
+ "model.layers.44.mlp.down_proj.weight": "pytorch_model-00017-of-00029.bin",
361
+ "model.layers.44.mlp.gate_proj.weight": "pytorch_model-00016-of-00029.bin",
362
+ "model.layers.44.mlp.up_proj.weight": "pytorch_model-00017-of-00029.bin",
363
+ "model.layers.44.post_attention_layernorm.weight": "pytorch_model-00017-of-00029.bin",
364
+ "model.layers.44.self_attn.k_proj.weight": "pytorch_model-00016-of-00029.bin",
365
+ "model.layers.44.self_attn.o_proj.weight": "pytorch_model-00016-of-00029.bin",
366
+ "model.layers.44.self_attn.q_proj.weight": "pytorch_model-00016-of-00029.bin",
367
+ "model.layers.44.self_attn.v_proj.weight": "pytorch_model-00016-of-00029.bin",
368
+ "model.layers.45.input_layernorm.weight": "pytorch_model-00017-of-00029.bin",
369
+ "model.layers.45.mlp.down_proj.weight": "pytorch_model-00017-of-00029.bin",
370
+ "model.layers.45.mlp.gate_proj.weight": "pytorch_model-00017-of-00029.bin",
371
+ "model.layers.45.mlp.up_proj.weight": "pytorch_model-00017-of-00029.bin",
372
+ "model.layers.45.post_attention_layernorm.weight": "pytorch_model-00017-of-00029.bin",
373
+ "model.layers.45.self_attn.k_proj.weight": "pytorch_model-00017-of-00029.bin",
374
+ "model.layers.45.self_attn.o_proj.weight": "pytorch_model-00017-of-00029.bin",
375
+ "model.layers.45.self_attn.q_proj.weight": "pytorch_model-00017-of-00029.bin",
376
+ "model.layers.45.self_attn.v_proj.weight": "pytorch_model-00017-of-00029.bin",
377
+ "model.layers.46.input_layernorm.weight": "pytorch_model-00017-of-00029.bin",
378
+ "model.layers.46.mlp.down_proj.weight": "pytorch_model-00017-of-00029.bin",
379
+ "model.layers.46.mlp.gate_proj.weight": "pytorch_model-00017-of-00029.bin",
380
+ "model.layers.46.mlp.up_proj.weight": "pytorch_model-00017-of-00029.bin",
381
+ "model.layers.46.post_attention_layernorm.weight": "pytorch_model-00017-of-00029.bin",
382
+ "model.layers.46.self_attn.k_proj.weight": "pytorch_model-00017-of-00029.bin",
383
+ "model.layers.46.self_attn.o_proj.weight": "pytorch_model-00017-of-00029.bin",
384
+ "model.layers.46.self_attn.q_proj.weight": "pytorch_model-00017-of-00029.bin",
385
+ "model.layers.46.self_attn.v_proj.weight": "pytorch_model-00017-of-00029.bin",
386
+ "model.layers.47.input_layernorm.weight": "pytorch_model-00018-of-00029.bin",
387
+ "model.layers.47.mlp.down_proj.weight": "pytorch_model-00018-of-00029.bin",
388
+ "model.layers.47.mlp.gate_proj.weight": "pytorch_model-00018-of-00029.bin",
389
+ "model.layers.47.mlp.up_proj.weight": "pytorch_model-00018-of-00029.bin",
390
+ "model.layers.47.post_attention_layernorm.weight": "pytorch_model-00018-of-00029.bin",
391
+ "model.layers.47.self_attn.k_proj.weight": "pytorch_model-00017-of-00029.bin",
392
+ "model.layers.47.self_attn.o_proj.weight": "pytorch_model-00017-of-00029.bin",
393
+ "model.layers.47.self_attn.q_proj.weight": "pytorch_model-00017-of-00029.bin",
394
+ "model.layers.47.self_attn.v_proj.weight": "pytorch_model-00017-of-00029.bin",
395
+ "model.layers.48.input_layernorm.weight": "pytorch_model-00018-of-00029.bin",
396
+ "model.layers.48.mlp.down_proj.weight": "pytorch_model-00018-of-00029.bin",
397
+ "model.layers.48.mlp.gate_proj.weight": "pytorch_model-00018-of-00029.bin",
398
+ "model.layers.48.mlp.up_proj.weight": "pytorch_model-00018-of-00029.bin",
399
+ "model.layers.48.post_attention_layernorm.weight": "pytorch_model-00018-of-00029.bin",
400
+ "model.layers.48.self_attn.k_proj.weight": "pytorch_model-00018-of-00029.bin",
401
+ "model.layers.48.self_attn.o_proj.weight": "pytorch_model-00018-of-00029.bin",
402
+ "model.layers.48.self_attn.q_proj.weight": "pytorch_model-00018-of-00029.bin",
403
+ "model.layers.48.self_attn.v_proj.weight": "pytorch_model-00018-of-00029.bin",
404
+ "model.layers.49.input_layernorm.weight": "pytorch_model-00018-of-00029.bin",
405
+ "model.layers.49.mlp.down_proj.weight": "pytorch_model-00018-of-00029.bin",
406
+ "model.layers.49.mlp.gate_proj.weight": "pytorch_model-00018-of-00029.bin",
407
+ "model.layers.49.mlp.up_proj.weight": "pytorch_model-00018-of-00029.bin",
408
+ "model.layers.49.post_attention_layernorm.weight": "pytorch_model-00018-of-00029.bin",
409
+ "model.layers.49.self_attn.k_proj.weight": "pytorch_model-00018-of-00029.bin",
410
+ "model.layers.49.self_attn.o_proj.weight": "pytorch_model-00018-of-00029.bin",
411
+ "model.layers.49.self_attn.q_proj.weight": "pytorch_model-00018-of-00029.bin",
412
+ "model.layers.49.self_attn.v_proj.weight": "pytorch_model-00018-of-00029.bin",
413
+ "model.layers.5.input_layernorm.weight": "pytorch_model-00003-of-00029.bin",
414
+ "model.layers.5.mlp.down_proj.weight": "pytorch_model-00003-of-00029.bin",
415
+ "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00003-of-00029.bin",
416
+ "model.layers.5.mlp.up_proj.weight": "pytorch_model-00003-of-00029.bin",
417
+ "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00003-of-00029.bin",
418
+ "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00002-of-00029.bin",
419
+ "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00002-of-00029.bin",
420
+ "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00002-of-00029.bin",
421
+ "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00002-of-00029.bin",
422
+ "model.layers.50.input_layernorm.weight": "pytorch_model-00019-of-00029.bin",
423
+ "model.layers.50.mlp.down_proj.weight": "pytorch_model-00019-of-00029.bin",
424
+ "model.layers.50.mlp.gate_proj.weight": "pytorch_model-00019-of-00029.bin",
425
+ "model.layers.50.mlp.up_proj.weight": "pytorch_model-00019-of-00029.bin",
426
+ "model.layers.50.post_attention_layernorm.weight": "pytorch_model-00019-of-00029.bin",
427
+ "model.layers.50.self_attn.k_proj.weight": "pytorch_model-00018-of-00029.bin",
428
+ "model.layers.50.self_attn.o_proj.weight": "pytorch_model-00019-of-00029.bin",
429
+ "model.layers.50.self_attn.q_proj.weight": "pytorch_model-00018-of-00029.bin",
430
+ "model.layers.50.self_attn.v_proj.weight": "pytorch_model-00018-of-00029.bin",
431
+ "model.layers.51.input_layernorm.weight": "pytorch_model-00019-of-00029.bin",
432
+ "model.layers.51.mlp.down_proj.weight": "pytorch_model-00019-of-00029.bin",
433
+ "model.layers.51.mlp.gate_proj.weight": "pytorch_model-00019-of-00029.bin",
434
+ "model.layers.51.mlp.up_proj.weight": "pytorch_model-00019-of-00029.bin",
435
+ "model.layers.51.post_attention_layernorm.weight": "pytorch_model-00019-of-00029.bin",
436
+ "model.layers.51.self_attn.k_proj.weight": "pytorch_model-00019-of-00029.bin",
437
+ "model.layers.51.self_attn.o_proj.weight": "pytorch_model-00019-of-00029.bin",
438
+ "model.layers.51.self_attn.q_proj.weight": "pytorch_model-00019-of-00029.bin",
439
+ "model.layers.51.self_attn.v_proj.weight": "pytorch_model-00019-of-00029.bin",
440
+ "model.layers.52.input_layernorm.weight": "pytorch_model-00019-of-00029.bin",
441
+ "model.layers.52.mlp.down_proj.weight": "pytorch_model-00019-of-00029.bin",
442
+ "model.layers.52.mlp.gate_proj.weight": "pytorch_model-00019-of-00029.bin",
443
+ "model.layers.52.mlp.up_proj.weight": "pytorch_model-00019-of-00029.bin",
444
+ "model.layers.52.post_attention_layernorm.weight": "pytorch_model-00019-of-00029.bin",
445
+ "model.layers.52.self_attn.k_proj.weight": "pytorch_model-00019-of-00029.bin",
446
+ "model.layers.52.self_attn.o_proj.weight": "pytorch_model-00019-of-00029.bin",
447
+ "model.layers.52.self_attn.q_proj.weight": "pytorch_model-00019-of-00029.bin",
448
+ "model.layers.52.self_attn.v_proj.weight": "pytorch_model-00019-of-00029.bin",
449
+ "model.layers.53.input_layernorm.weight": "pytorch_model-00020-of-00029.bin",
450
+ "model.layers.53.mlp.down_proj.weight": "pytorch_model-00020-of-00029.bin",
451
+ "model.layers.53.mlp.gate_proj.weight": "pytorch_model-00020-of-00029.bin",
452
+ "model.layers.53.mlp.up_proj.weight": "pytorch_model-00020-of-00029.bin",
453
+ "model.layers.53.post_attention_layernorm.weight": "pytorch_model-00020-of-00029.bin",
454
+ "model.layers.53.self_attn.k_proj.weight": "pytorch_model-00020-of-00029.bin",
455
+ "model.layers.53.self_attn.o_proj.weight": "pytorch_model-00020-of-00029.bin",
456
+ "model.layers.53.self_attn.q_proj.weight": "pytorch_model-00020-of-00029.bin",
457
+ "model.layers.53.self_attn.v_proj.weight": "pytorch_model-00020-of-00029.bin",
458
+ "model.layers.54.input_layernorm.weight": "pytorch_model-00020-of-00029.bin",
459
+ "model.layers.54.mlp.down_proj.weight": "pytorch_model-00020-of-00029.bin",
460
+ "model.layers.54.mlp.gate_proj.weight": "pytorch_model-00020-of-00029.bin",
461
+ "model.layers.54.mlp.up_proj.weight": "pytorch_model-00020-of-00029.bin",
462
+ "model.layers.54.post_attention_layernorm.weight": "pytorch_model-00020-of-00029.bin",
463
+ "model.layers.54.self_attn.k_proj.weight": "pytorch_model-00020-of-00029.bin",
464
+ "model.layers.54.self_attn.o_proj.weight": "pytorch_model-00020-of-00029.bin",
465
+ "model.layers.54.self_attn.q_proj.weight": "pytorch_model-00020-of-00029.bin",
466
+ "model.layers.54.self_attn.v_proj.weight": "pytorch_model-00020-of-00029.bin",
467
+ "model.layers.55.input_layernorm.weight": "pytorch_model-00021-of-00029.bin",
468
+ "model.layers.55.mlp.down_proj.weight": "pytorch_model-00021-of-00029.bin",
469
+ "model.layers.55.mlp.gate_proj.weight": "pytorch_model-00020-of-00029.bin",
470
+ "model.layers.55.mlp.up_proj.weight": "pytorch_model-00020-of-00029.bin",
471
+ "model.layers.55.post_attention_layernorm.weight": "pytorch_model-00021-of-00029.bin",
472
+ "model.layers.55.self_attn.k_proj.weight": "pytorch_model-00020-of-00029.bin",
473
+ "model.layers.55.self_attn.o_proj.weight": "pytorch_model-00020-of-00029.bin",
474
+ "model.layers.55.self_attn.q_proj.weight": "pytorch_model-00020-of-00029.bin",
475
+ "model.layers.55.self_attn.v_proj.weight": "pytorch_model-00020-of-00029.bin",
476
+ "model.layers.56.input_layernorm.weight": "pytorch_model-00021-of-00029.bin",
477
+ "model.layers.56.mlp.down_proj.weight": "pytorch_model-00021-of-00029.bin",
478
+ "model.layers.56.mlp.gate_proj.weight": "pytorch_model-00021-of-00029.bin",
479
+ "model.layers.56.mlp.up_proj.weight": "pytorch_model-00021-of-00029.bin",
480
+ "model.layers.56.post_attention_layernorm.weight": "pytorch_model-00021-of-00029.bin",
481
+ "model.layers.56.self_attn.k_proj.weight": "pytorch_model-00021-of-00029.bin",
482
+ "model.layers.56.self_attn.o_proj.weight": "pytorch_model-00021-of-00029.bin",
483
+ "model.layers.56.self_attn.q_proj.weight": "pytorch_model-00021-of-00029.bin",
484
+ "model.layers.56.self_attn.v_proj.weight": "pytorch_model-00021-of-00029.bin",
485
+ "model.layers.57.input_layernorm.weight": "pytorch_model-00021-of-00029.bin",
486
+ "model.layers.57.mlp.down_proj.weight": "pytorch_model-00021-of-00029.bin",
487
+ "model.layers.57.mlp.gate_proj.weight": "pytorch_model-00021-of-00029.bin",
488
+ "model.layers.57.mlp.up_proj.weight": "pytorch_model-00021-of-00029.bin",
489
+ "model.layers.57.post_attention_layernorm.weight": "pytorch_model-00021-of-00029.bin",
490
+ "model.layers.57.self_attn.k_proj.weight": "pytorch_model-00021-of-00029.bin",
491
+ "model.layers.57.self_attn.o_proj.weight": "pytorch_model-00021-of-00029.bin",
492
+ "model.layers.57.self_attn.q_proj.weight": "pytorch_model-00021-of-00029.bin",
493
+ "model.layers.57.self_attn.v_proj.weight": "pytorch_model-00021-of-00029.bin",
494
+ "model.layers.58.input_layernorm.weight": "pytorch_model-00022-of-00029.bin",
495
+ "model.layers.58.mlp.down_proj.weight": "pytorch_model-00022-of-00029.bin",
496
+ "model.layers.58.mlp.gate_proj.weight": "pytorch_model-00021-of-00029.bin",
497
+ "model.layers.58.mlp.up_proj.weight": "pytorch_model-00022-of-00029.bin",
498
+ "model.layers.58.post_attention_layernorm.weight": "pytorch_model-00022-of-00029.bin",
499
+ "model.layers.58.self_attn.k_proj.weight": "pytorch_model-00021-of-00029.bin",
500
+ "model.layers.58.self_attn.o_proj.weight": "pytorch_model-00021-of-00029.bin",
501
+ "model.layers.58.self_attn.q_proj.weight": "pytorch_model-00021-of-00029.bin",
502
+ "model.layers.58.self_attn.v_proj.weight": "pytorch_model-00021-of-00029.bin",
503
+ "model.layers.59.input_layernorm.weight": "pytorch_model-00022-of-00029.bin",
504
+ "model.layers.59.mlp.down_proj.weight": "pytorch_model-00022-of-00029.bin",
505
+ "model.layers.59.mlp.gate_proj.weight": "pytorch_model-00022-of-00029.bin",
506
+ "model.layers.59.mlp.up_proj.weight": "pytorch_model-00022-of-00029.bin",
507
+ "model.layers.59.post_attention_layernorm.weight": "pytorch_model-00022-of-00029.bin",
508
+ "model.layers.59.self_attn.k_proj.weight": "pytorch_model-00022-of-00029.bin",
509
+ "model.layers.59.self_attn.o_proj.weight": "pytorch_model-00022-of-00029.bin",
510
+ "model.layers.59.self_attn.q_proj.weight": "pytorch_model-00022-of-00029.bin",
511
+ "model.layers.59.self_attn.v_proj.weight": "pytorch_model-00022-of-00029.bin",
512
+ "model.layers.6.input_layernorm.weight": "pytorch_model-00003-of-00029.bin",
513
+ "model.layers.6.mlp.down_proj.weight": "pytorch_model-00003-of-00029.bin",
514
+ "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00003-of-00029.bin",
515
+ "model.layers.6.mlp.up_proj.weight": "pytorch_model-00003-of-00029.bin",
516
+ "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00003-of-00029.bin",
517
+ "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00003-of-00029.bin",
518
+ "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00003-of-00029.bin",
519
+ "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00003-of-00029.bin",
520
+ "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00003-of-00029.bin",
521
+ "model.layers.60.input_layernorm.weight": "pytorch_model-00022-of-00029.bin",
522
+ "model.layers.60.mlp.down_proj.weight": "pytorch_model-00022-of-00029.bin",
523
+ "model.layers.60.mlp.gate_proj.weight": "pytorch_model-00022-of-00029.bin",
524
+ "model.layers.60.mlp.up_proj.weight": "pytorch_model-00022-of-00029.bin",
525
+ "model.layers.60.post_attention_layernorm.weight": "pytorch_model-00022-of-00029.bin",
526
+ "model.layers.60.self_attn.k_proj.weight": "pytorch_model-00022-of-00029.bin",
527
+ "model.layers.60.self_attn.o_proj.weight": "pytorch_model-00022-of-00029.bin",
528
+ "model.layers.60.self_attn.q_proj.weight": "pytorch_model-00022-of-00029.bin",
529
+ "model.layers.60.self_attn.v_proj.weight": "pytorch_model-00022-of-00029.bin",
530
+ "model.layers.61.input_layernorm.weight": "pytorch_model-00023-of-00029.bin",
531
+ "model.layers.61.mlp.down_proj.weight": "pytorch_model-00023-of-00029.bin",
532
+ "model.layers.61.mlp.gate_proj.weight": "pytorch_model-00023-of-00029.bin",
533
+ "model.layers.61.mlp.up_proj.weight": "pytorch_model-00023-of-00029.bin",
534
+ "model.layers.61.post_attention_layernorm.weight": "pytorch_model-00023-of-00029.bin",
535
+ "model.layers.61.self_attn.k_proj.weight": "pytorch_model-00022-of-00029.bin",
536
+ "model.layers.61.self_attn.o_proj.weight": "pytorch_model-00022-of-00029.bin",
537
+ "model.layers.61.self_attn.q_proj.weight": "pytorch_model-00022-of-00029.bin",
538
+ "model.layers.61.self_attn.v_proj.weight": "pytorch_model-00022-of-00029.bin",
539
+ "model.layers.62.input_layernorm.weight": "pytorch_model-00023-of-00029.bin",
540
+ "model.layers.62.mlp.down_proj.weight": "pytorch_model-00023-of-00029.bin",
541
+ "model.layers.62.mlp.gate_proj.weight": "pytorch_model-00023-of-00029.bin",
542
+ "model.layers.62.mlp.up_proj.weight": "pytorch_model-00023-of-00029.bin",
543
+ "model.layers.62.post_attention_layernorm.weight": "pytorch_model-00023-of-00029.bin",
544
+ "model.layers.62.self_attn.k_proj.weight": "pytorch_model-00023-of-00029.bin",
545
+ "model.layers.62.self_attn.o_proj.weight": "pytorch_model-00023-of-00029.bin",
546
+ "model.layers.62.self_attn.q_proj.weight": "pytorch_model-00023-of-00029.bin",
547
+ "model.layers.62.self_attn.v_proj.weight": "pytorch_model-00023-of-00029.bin",
548
+ "model.layers.63.input_layernorm.weight": "pytorch_model-00023-of-00029.bin",
549
+ "model.layers.63.mlp.down_proj.weight": "pytorch_model-00023-of-00029.bin",
550
+ "model.layers.63.mlp.gate_proj.weight": "pytorch_model-00023-of-00029.bin",
551
+ "model.layers.63.mlp.up_proj.weight": "pytorch_model-00023-of-00029.bin",
552
+ "model.layers.63.post_attention_layernorm.weight": "pytorch_model-00023-of-00029.bin",
553
+ "model.layers.63.self_attn.k_proj.weight": "pytorch_model-00023-of-00029.bin",
554
+ "model.layers.63.self_attn.o_proj.weight": "pytorch_model-00023-of-00029.bin",
555
+ "model.layers.63.self_attn.q_proj.weight": "pytorch_model-00023-of-00029.bin",
556
+ "model.layers.63.self_attn.v_proj.weight": "pytorch_model-00023-of-00029.bin",
557
+ "model.layers.64.input_layernorm.weight": "pytorch_model-00024-of-00029.bin",
558
+ "model.layers.64.mlp.down_proj.weight": "pytorch_model-00024-of-00029.bin",
559
+ "model.layers.64.mlp.gate_proj.weight": "pytorch_model-00024-of-00029.bin",
560
+ "model.layers.64.mlp.up_proj.weight": "pytorch_model-00024-of-00029.bin",
561
+ "model.layers.64.post_attention_layernorm.weight": "pytorch_model-00024-of-00029.bin",
562
+ "model.layers.64.self_attn.k_proj.weight": "pytorch_model-00023-of-00029.bin",
563
+ "model.layers.64.self_attn.o_proj.weight": "pytorch_model-00024-of-00029.bin",
564
+ "model.layers.64.self_attn.q_proj.weight": "pytorch_model-00023-of-00029.bin",
565
+ "model.layers.64.self_attn.v_proj.weight": "pytorch_model-00023-of-00029.bin",
566
+ "model.layers.65.input_layernorm.weight": "pytorch_model-00024-of-00029.bin",
567
+ "model.layers.65.mlp.down_proj.weight": "pytorch_model-00024-of-00029.bin",
568
+ "model.layers.65.mlp.gate_proj.weight": "pytorch_model-00024-of-00029.bin",
569
+ "model.layers.65.mlp.up_proj.weight": "pytorch_model-00024-of-00029.bin",
570
+ "model.layers.65.post_attention_layernorm.weight": "pytorch_model-00024-of-00029.bin",
571
+ "model.layers.65.self_attn.k_proj.weight": "pytorch_model-00024-of-00029.bin",
572
+ "model.layers.65.self_attn.o_proj.weight": "pytorch_model-00024-of-00029.bin",
573
+ "model.layers.65.self_attn.q_proj.weight": "pytorch_model-00024-of-00029.bin",
574
+ "model.layers.65.self_attn.v_proj.weight": "pytorch_model-00024-of-00029.bin",
575
+ "model.layers.66.input_layernorm.weight": "pytorch_model-00024-of-00029.bin",
576
+ "model.layers.66.mlp.down_proj.weight": "pytorch_model-00024-of-00029.bin",
577
+ "model.layers.66.mlp.gate_proj.weight": "pytorch_model-00024-of-00029.bin",
578
+ "model.layers.66.mlp.up_proj.weight": "pytorch_model-00024-of-00029.bin",
579
+ "model.layers.66.post_attention_layernorm.weight": "pytorch_model-00024-of-00029.bin",
580
+ "model.layers.66.self_attn.k_proj.weight": "pytorch_model-00024-of-00029.bin",
581
+ "model.layers.66.self_attn.o_proj.weight": "pytorch_model-00024-of-00029.bin",
582
+ "model.layers.66.self_attn.q_proj.weight": "pytorch_model-00024-of-00029.bin",
583
+ "model.layers.66.self_attn.v_proj.weight": "pytorch_model-00024-of-00029.bin",
584
+ "model.layers.67.input_layernorm.weight": "pytorch_model-00025-of-00029.bin",
585
+ "model.layers.67.mlp.down_proj.weight": "pytorch_model-00025-of-00029.bin",
586
+ "model.layers.67.mlp.gate_proj.weight": "pytorch_model-00025-of-00029.bin",
587
+ "model.layers.67.mlp.up_proj.weight": "pytorch_model-00025-of-00029.bin",
588
+ "model.layers.67.post_attention_layernorm.weight": "pytorch_model-00025-of-00029.bin",
589
+ "model.layers.67.self_attn.k_proj.weight": "pytorch_model-00025-of-00029.bin",
590
+ "model.layers.67.self_attn.o_proj.weight": "pytorch_model-00025-of-00029.bin",
591
+ "model.layers.67.self_attn.q_proj.weight": "pytorch_model-00025-of-00029.bin",
592
+ "model.layers.67.self_attn.v_proj.weight": "pytorch_model-00025-of-00029.bin",
593
+ "model.layers.68.input_layernorm.weight": "pytorch_model-00025-of-00029.bin",
594
+ "model.layers.68.mlp.down_proj.weight": "pytorch_model-00025-of-00029.bin",
595
+ "model.layers.68.mlp.gate_proj.weight": "pytorch_model-00025-of-00029.bin",
596
+ "model.layers.68.mlp.up_proj.weight": "pytorch_model-00025-of-00029.bin",
597
+ "model.layers.68.post_attention_layernorm.weight": "pytorch_model-00025-of-00029.bin",
598
+ "model.layers.68.self_attn.k_proj.weight": "pytorch_model-00025-of-00029.bin",
599
+ "model.layers.68.self_attn.o_proj.weight": "pytorch_model-00025-of-00029.bin",
600
+ "model.layers.68.self_attn.q_proj.weight": "pytorch_model-00025-of-00029.bin",
601
+ "model.layers.68.self_attn.v_proj.weight": "pytorch_model-00025-of-00029.bin",
602
+ "model.layers.69.input_layernorm.weight": "pytorch_model-00026-of-00029.bin",
603
+ "model.layers.69.mlp.down_proj.weight": "pytorch_model-00026-of-00029.bin",
604
+ "model.layers.69.mlp.gate_proj.weight": "pytorch_model-00025-of-00029.bin",
605
+ "model.layers.69.mlp.up_proj.weight": "pytorch_model-00025-of-00029.bin",
606
+ "model.layers.69.post_attention_layernorm.weight": "pytorch_model-00026-of-00029.bin",
607
+ "model.layers.69.self_attn.k_proj.weight": "pytorch_model-00025-of-00029.bin",
608
+ "model.layers.69.self_attn.o_proj.weight": "pytorch_model-00025-of-00029.bin",
609
+ "model.layers.69.self_attn.q_proj.weight": "pytorch_model-00025-of-00029.bin",
610
+ "model.layers.69.self_attn.v_proj.weight": "pytorch_model-00025-of-00029.bin",
611
+ "model.layers.7.input_layernorm.weight": "pytorch_model-00003-of-00029.bin",
612
+ "model.layers.7.mlp.down_proj.weight": "pytorch_model-00003-of-00029.bin",
613
+ "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00003-of-00029.bin",
614
+ "model.layers.7.mlp.up_proj.weight": "pytorch_model-00003-of-00029.bin",
615
+ "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00003-of-00029.bin",
616
+ "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00003-of-00029.bin",
617
+ "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00003-of-00029.bin",
618
+ "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00003-of-00029.bin",
619
+ "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00003-of-00029.bin",
620
+ "model.layers.70.input_layernorm.weight": "pytorch_model-00026-of-00029.bin",
621
+ "model.layers.70.mlp.down_proj.weight": "pytorch_model-00026-of-00029.bin",
622
+ "model.layers.70.mlp.gate_proj.weight": "pytorch_model-00026-of-00029.bin",
623
+ "model.layers.70.mlp.up_proj.weight": "pytorch_model-00026-of-00029.bin",
624
+ "model.layers.70.post_attention_layernorm.weight": "pytorch_model-00026-of-00029.bin",
625
+ "model.layers.70.self_attn.k_proj.weight": "pytorch_model-00026-of-00029.bin",
626
+ "model.layers.70.self_attn.o_proj.weight": "pytorch_model-00026-of-00029.bin",
627
+ "model.layers.70.self_attn.q_proj.weight": "pytorch_model-00026-of-00029.bin",
628
+ "model.layers.70.self_attn.v_proj.weight": "pytorch_model-00026-of-00029.bin",
629
+ "model.layers.71.input_layernorm.weight": "pytorch_model-00026-of-00029.bin",
630
+ "model.layers.71.mlp.down_proj.weight": "pytorch_model-00026-of-00029.bin",
631
+ "model.layers.71.mlp.gate_proj.weight": "pytorch_model-00026-of-00029.bin",
632
+ "model.layers.71.mlp.up_proj.weight": "pytorch_model-00026-of-00029.bin",
633
+ "model.layers.71.post_attention_layernorm.weight": "pytorch_model-00026-of-00029.bin",
634
+ "model.layers.71.self_attn.k_proj.weight": "pytorch_model-00026-of-00029.bin",
635
+ "model.layers.71.self_attn.o_proj.weight": "pytorch_model-00026-of-00029.bin",
636
+ "model.layers.71.self_attn.q_proj.weight": "pytorch_model-00026-of-00029.bin",
637
+ "model.layers.71.self_attn.v_proj.weight": "pytorch_model-00026-of-00029.bin",
638
+ "model.layers.72.input_layernorm.weight": "pytorch_model-00027-of-00029.bin",
639
+ "model.layers.72.mlp.down_proj.weight": "pytorch_model-00027-of-00029.bin",
640
+ "model.layers.72.mlp.gate_proj.weight": "pytorch_model-00026-of-00029.bin",
641
+ "model.layers.72.mlp.up_proj.weight": "pytorch_model-00027-of-00029.bin",
642
+ "model.layers.72.post_attention_layernorm.weight": "pytorch_model-00027-of-00029.bin",
643
+ "model.layers.72.self_attn.k_proj.weight": "pytorch_model-00026-of-00029.bin",
644
+ "model.layers.72.self_attn.o_proj.weight": "pytorch_model-00026-of-00029.bin",
645
+ "model.layers.72.self_attn.q_proj.weight": "pytorch_model-00026-of-00029.bin",
646
+ "model.layers.72.self_attn.v_proj.weight": "pytorch_model-00026-of-00029.bin",
647
+ "model.layers.73.input_layernorm.weight": "pytorch_model-00027-of-00029.bin",
648
+ "model.layers.73.mlp.down_proj.weight": "pytorch_model-00027-of-00029.bin",
649
+ "model.layers.73.mlp.gate_proj.weight": "pytorch_model-00027-of-00029.bin",
650
+ "model.layers.73.mlp.up_proj.weight": "pytorch_model-00027-of-00029.bin",
651
+ "model.layers.73.post_attention_layernorm.weight": "pytorch_model-00027-of-00029.bin",
652
+ "model.layers.73.self_attn.k_proj.weight": "pytorch_model-00027-of-00029.bin",
653
+ "model.layers.73.self_attn.o_proj.weight": "pytorch_model-00027-of-00029.bin",
654
+ "model.layers.73.self_attn.q_proj.weight": "pytorch_model-00027-of-00029.bin",
655
+ "model.layers.73.self_attn.v_proj.weight": "pytorch_model-00027-of-00029.bin",
656
+ "model.layers.74.input_layernorm.weight": "pytorch_model-00027-of-00029.bin",
657
+ "model.layers.74.mlp.down_proj.weight": "pytorch_model-00027-of-00029.bin",
658
+ "model.layers.74.mlp.gate_proj.weight": "pytorch_model-00027-of-00029.bin",
659
+ "model.layers.74.mlp.up_proj.weight": "pytorch_model-00027-of-00029.bin",
660
+ "model.layers.74.post_attention_layernorm.weight": "pytorch_model-00027-of-00029.bin",
661
+ "model.layers.74.self_attn.k_proj.weight": "pytorch_model-00027-of-00029.bin",
662
+ "model.layers.74.self_attn.o_proj.weight": "pytorch_model-00027-of-00029.bin",
663
+ "model.layers.74.self_attn.q_proj.weight": "pytorch_model-00027-of-00029.bin",
664
+ "model.layers.74.self_attn.v_proj.weight": "pytorch_model-00027-of-00029.bin",
665
+ "model.layers.75.input_layernorm.weight": "pytorch_model-00028-of-00029.bin",
666
+ "model.layers.75.mlp.down_proj.weight": "pytorch_model-00028-of-00029.bin",
667
+ "model.layers.75.mlp.gate_proj.weight": "pytorch_model-00028-of-00029.bin",
668
+ "model.layers.75.mlp.up_proj.weight": "pytorch_model-00028-of-00029.bin",
669
+ "model.layers.75.post_attention_layernorm.weight": "pytorch_model-00028-of-00029.bin",
670
+ "model.layers.75.self_attn.k_proj.weight": "pytorch_model-00027-of-00029.bin",
671
+ "model.layers.75.self_attn.o_proj.weight": "pytorch_model-00027-of-00029.bin",
672
+ "model.layers.75.self_attn.q_proj.weight": "pytorch_model-00027-of-00029.bin",
673
+ "model.layers.75.self_attn.v_proj.weight": "pytorch_model-00027-of-00029.bin",
674
+ "model.layers.76.input_layernorm.weight": "pytorch_model-00028-of-00029.bin",
675
+ "model.layers.76.mlp.down_proj.weight": "pytorch_model-00028-of-00029.bin",
676
+ "model.layers.76.mlp.gate_proj.weight": "pytorch_model-00028-of-00029.bin",
677
+ "model.layers.76.mlp.up_proj.weight": "pytorch_model-00028-of-00029.bin",
678
+ "model.layers.76.post_attention_layernorm.weight": "pytorch_model-00028-of-00029.bin",
679
+ "model.layers.76.self_attn.k_proj.weight": "pytorch_model-00028-of-00029.bin",
680
+ "model.layers.76.self_attn.o_proj.weight": "pytorch_model-00028-of-00029.bin",
681
+ "model.layers.76.self_attn.q_proj.weight": "pytorch_model-00028-of-00029.bin",
682
+ "model.layers.76.self_attn.v_proj.weight": "pytorch_model-00028-of-00029.bin",
683
+ "model.layers.77.input_layernorm.weight": "pytorch_model-00028-of-00029.bin",
684
+ "model.layers.77.mlp.down_proj.weight": "pytorch_model-00028-of-00029.bin",
685
+ "model.layers.77.mlp.gate_proj.weight": "pytorch_model-00028-of-00029.bin",
686
+ "model.layers.77.mlp.up_proj.weight": "pytorch_model-00028-of-00029.bin",
687
+ "model.layers.77.post_attention_layernorm.weight": "pytorch_model-00028-of-00029.bin",
688
+ "model.layers.77.self_attn.k_proj.weight": "pytorch_model-00028-of-00029.bin",
689
+ "model.layers.77.self_attn.o_proj.weight": "pytorch_model-00028-of-00029.bin",
690
+ "model.layers.77.self_attn.q_proj.weight": "pytorch_model-00028-of-00029.bin",
691
+ "model.layers.77.self_attn.v_proj.weight": "pytorch_model-00028-of-00029.bin",
692
+ "model.layers.78.input_layernorm.weight": "pytorch_model-00029-of-00029.bin",
693
+ "model.layers.78.mlp.down_proj.weight": "pytorch_model-00029-of-00029.bin",
694
+ "model.layers.78.mlp.gate_proj.weight": "pytorch_model-00029-of-00029.bin",
695
+ "model.layers.78.mlp.up_proj.weight": "pytorch_model-00029-of-00029.bin",
696
+ "model.layers.78.post_attention_layernorm.weight": "pytorch_model-00029-of-00029.bin",
697
+ "model.layers.78.self_attn.k_proj.weight": "pytorch_model-00028-of-00029.bin",
698
+ "model.layers.78.self_attn.o_proj.weight": "pytorch_model-00029-of-00029.bin",
699
+ "model.layers.78.self_attn.q_proj.weight": "pytorch_model-00028-of-00029.bin",
700
+ "model.layers.78.self_attn.v_proj.weight": "pytorch_model-00028-of-00029.bin",
701
+ "model.layers.79.input_layernorm.weight": "pytorch_model-00029-of-00029.bin",
702
+ "model.layers.79.mlp.down_proj.weight": "pytorch_model-00029-of-00029.bin",
703
+ "model.layers.79.mlp.gate_proj.weight": "pytorch_model-00029-of-00029.bin",
704
+ "model.layers.79.mlp.up_proj.weight": "pytorch_model-00029-of-00029.bin",
705
+ "model.layers.79.post_attention_layernorm.weight": "pytorch_model-00029-of-00029.bin",
706
+ "model.layers.79.self_attn.k_proj.weight": "pytorch_model-00029-of-00029.bin",
707
+ "model.layers.79.self_attn.o_proj.weight": "pytorch_model-00029-of-00029.bin",
708
+ "model.layers.79.self_attn.q_proj.weight": "pytorch_model-00029-of-00029.bin",
709
+ "model.layers.79.self_attn.v_proj.weight": "pytorch_model-00029-of-00029.bin",
710
+ "model.layers.8.input_layernorm.weight": "pytorch_model-00004-of-00029.bin",
711
+ "model.layers.8.mlp.down_proj.weight": "pytorch_model-00004-of-00029.bin",
712
+ "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00004-of-00029.bin",
713
+ "model.layers.8.mlp.up_proj.weight": "pytorch_model-00004-of-00029.bin",
714
+ "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00004-of-00029.bin",
715
+ "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00003-of-00029.bin",
716
+ "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00004-of-00029.bin",
717
+ "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00003-of-00029.bin",
718
+ "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00003-of-00029.bin",
719
+ "model.layers.9.input_layernorm.weight": "pytorch_model-00004-of-00029.bin",
720
+ "model.layers.9.mlp.down_proj.weight": "pytorch_model-00004-of-00029.bin",
721
+ "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00004-of-00029.bin",
722
+ "model.layers.9.mlp.up_proj.weight": "pytorch_model-00004-of-00029.bin",
723
+ "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00004-of-00029.bin",
724
+ "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00004-of-00029.bin",
725
+ "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00004-of-00029.bin",
726
+ "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00004-of-00029.bin",
727
+ "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00004-of-00029.bin",
728
+ "model.norm.weight": "pytorch_model-00029-of-00029.bin"
729
+ }
730
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "<|im_start|>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "<|im_end|>",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ }
17
+ ],
18
+ "bos_token": {
19
+ "content": "<s>",
20
+ "lstrip": false,
21
+ "normalized": false,
22
+ "rstrip": false,
23
+ "single_word": false
24
+ },
25
+ "eos_token": {
26
+ "content": "</s>",
27
+ "lstrip": false,
28
+ "normalized": false,
29
+ "rstrip": false,
30
+ "single_word": false
31
+ },
32
+ "pad_token": "</s>",
33
+ "unk_token": {
34
+ "content": "<unk>",
35
+ "lstrip": false,
36
+ "normalized": false,
37
+ "rstrip": false,
38
+ "single_word": false
39
+ }
40
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
tokenizer_config.json ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "add_prefix_space": true,
5
+ "added_tokens_decoder": {
6
+ "0": {
7
+ "content": "<unk>",
8
+ "lstrip": false,
9
+ "normalized": false,
10
+ "rstrip": false,
11
+ "single_word": false,
12
+ "special": true
13
+ },
14
+ "1": {
15
+ "content": "<s>",
16
+ "lstrip": false,
17
+ "normalized": false,
18
+ "rstrip": false,
19
+ "single_word": false,
20
+ "special": true
21
+ },
22
+ "2": {
23
+ "content": "</s>",
24
+ "lstrip": false,
25
+ "normalized": false,
26
+ "rstrip": false,
27
+ "single_word": false,
28
+ "special": true
29
+ },
30
+ "32000": {
31
+ "content": "<|im_start|>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false,
36
+ "special": true
37
+ },
38
+ "32001": {
39
+ "content": "<|im_end|>",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": false,
43
+ "single_word": false,
44
+ "special": true
45
+ }
46
+ },
47
+ "additional_special_tokens": [
48
+ "<|im_start|>",
49
+ "<|im_end|>"
50
+ ],
51
+ "bos_token": "<s>",
52
+ "clean_up_tokenization_spaces": false,
53
+ "eos_token": "</s>",
54
+ "legacy": false,
55
+ "model_max_length": 1000000000000000019884624838656,
56
+ "pad_token": "</s>",
57
+ "padding_side": "right",
58
+ "sp_model_kwargs": {},
59
+ "spaces_between_special_tokens": false,
60
+ "tokenizer_class": "LlamaTokenizer",
61
+ "unk_token": "<unk>",
62
+ "use_default_system_prompt": false,
63
+ "use_fast": true
64
+ }