Upload folder using huggingface_hub

README.md

@@ -0,0 +1,82 @@
+---
+language:
+- code
+pipeline_tag: text-generation
+tags:
+- llama-2
+license: llama2
+---
+# **Code Llama**
+Code Llama is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the base 70B version in the Hugging Face Transformers format. This model is designed for general code synthesis and understanding. Links to other models can be found in the index at the bottom.
+
+|     | Base Model                                                                    | Python                                                                                      | Instruct                                                                                        |
+| --- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
+| 7B  | [codellama/CodeLlama-7b-hf](https://huggingface.co/codellama/CodeLlama-7b-hf) | [codellama/CodeLlama-7b-Python-hf](https://huggingface.co/codellama/CodeLlama-7b-Python-hf) | [codellama/CodeLlama-7b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf) |
+| 13B  | [codellama/CodeLlama-13b-hf](https://huggingface.co/codellama/CodeLlama-13b-hf) | [codellama/CodeLlama-13b-Python-hf](https://huggingface.co/codellama/CodeLlama-13b-Python-hf) | [codellama/CodeLlama-13b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-13b-Instruct-hf) |
+| 34B  | [codellama/CodeLlama-34b-hf](https://huggingface.co/codellama/CodeLlama-34b-hf) | [codellama/CodeLlama-34b-Python-hf](https://huggingface.co/codellama/CodeLlama-34b-Python-hf) | [codellama/CodeLlama-34b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf) |
+| 70B  | [codellama/CodeLlama-70b-hf](https://huggingface.co/codellama/CodeLlama-70b-hf) | [codellama/CodeLlama-70b-Python-hf](https://huggingface.co/codellama/CodeLlama-70b-Python-hf) | [codellama/CodeLlama-70b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-70b-Instruct-hf) |
+
+## Model Use
+
+To use this model, please make sure to install `transformers`.
+
+```bash
+pip install transformers accelerate
+```
+
+Model capabilities: 
+
+- [x] Code completion.
+- [ ] Infilling.
+- [ ] Instructions / chat.
+- [ ] Python specialist.
+
+## Model Details
+*Note: Use of this model is governed by the Meta license. Meta developed and publicly released the Code Llama family of large language models (LLMs).
+
+**Model Developers** Meta
+
+**Variations** Code Llama comes in four model sizes, and three variants:
+
+* Code Llama: base models designed for general code synthesis and understanding
+* Code Llama - Python: designed specifically for Python
+* Code Llama - Instruct: for instruction following and safer deployment
+
+All variants are available in sizes of 7B, 13B, 34B, and 70B parameters.
+
+**This repository contains the base version of the 70B parameters model.**
+
+**Input** Models input text only.
+
+**Output** Models generate text only.
+
+**Model Architecture** Code Llama is an auto-regressive language model that uses an optimized transformer architecture. It was fine-tuned with up to 16k tokens and supports up to 100k tokens at inference time.
+
+**Model Dates** Code Llama and its variants have been trained between January 2023 and January 2024.
+
+**Status** This is a static model trained on an offline dataset. Future versions of Code Llama - Instruct will be released as we improve model safety with community feedback.
+
+**License** A custom commercial license is available at: [https://ai.meta.com/resources/models-and-libraries/llama-downloads/](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)
+
+**Research Paper** More information can be found in the paper "[Code Llama: Open Foundation Models for Code](https://ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/)" or its [arXiv page](https://arxiv.org/abs/2308.12950).
+
+## Intended Use
+**Intended Use Cases** Code Llama and its variants are intended for commercial and research use in English and relevant programming languages. The base model Code Llama can be adapted for a variety of code synthesis and understanding tasks, Code Llama - Python is designed specifically to handle the Python programming language, and Code Llama - Instruct is intended to be safer to use for code assistant and generation applications.
+
+**Out-of-Scope Uses** Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in languages other than English. Use in any other way that is prohibited by the Acceptable Use Policy and Licensing Agreement for Code Llama and its variants.
+
+## Hardware and Software
+**Training Factors** We used custom training libraries. The training and fine-tuning of the released models have been performed Meta’s Research Super Cluster.
+
+**Carbon Footprint** In aggregate, training all 12 Code Llama models required 1400K GPU hours of computation on hardware of type A100-80GB (TDP of 350-400W). Estimated total emissions were 228.55 tCO2eq, 100% of which were offset by Meta’s sustainability program.
+
+
+## Evaluation Results
+
+See evaluations for the main models and detailed ablations in Section 3 and safety evaluations in Section 4 of the research paper.
+
+## Ethical Considerations and Limitations
+
+Code Llama and its variants are a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Code Llama’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. Therefore, before deploying any applications of Code Llama, developers should perform safety testing and tuning tailored to their specific applications of the model.
+
+Please see the Responsible Use Guide available available at [https://ai.meta.com/llama/responsible-use-guide](https://ai.meta.com/llama/responsible-use-guide).

config.json

@@ -0,0 +1,27 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 16384,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.37.1",
+  "use_cache": true,
+  "vocab_size": 32016
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.37.1"
+}

huggingface-metadata.txt

@@ -0,0 +1,63 @@
+url: https://huggingface.co/codellama/CodeLlama-70b-hf
+branch: main
+download date: 2024-01-29 23:34:33
+sha256sum:
+    c0f2626f33ef191c1914f00feff35b434a9fcb508cfa71aee92a985254ad9393 model-00001-of-00029.safetensors
+    fae0fe39fa1e8a3453a921554de3c463ca5a1330c32afd28735831c4650527af model-00002-of-00029.safetensors
+    24043e0618471611148167737324f50f98ca21552dbdd8f7d2649ef83ba7d216 model-00003-of-00029.safetensors
+    dfd7141cc091aa0c39bd907403d9c3730194bc2b7f0b138c200abbf2123da7d4 model-00004-of-00029.safetensors
+    7d2c9cede70040f2be38cc173b2d74bc14e2d3c479e183c5119add4455d55e0a model-00005-of-00029.safetensors
+    375e5a4b5f3a1be7efd46912b03ea19c55d40766439803962a5d3ac4efd0c567 model-00006-of-00029.safetensors
+    7341f8fc9eab124d6fea5f7b3b0a58402adc506a46bf1fc74c62106163e2a074 model-00007-of-00029.safetensors
+    56c77a9c527f5feb2a1469c933986f812a9687e10d206907e7ab846587d59333 model-00008-of-00029.safetensors
+    fe1df241b1b7347b5c840616db6c059e129f573033a6f8a09b8b3b94a4c8725b model-00009-of-00029.safetensors
+    aed80ac49b246100d99532ebc45a55ef344f84d17a4b5366ea9d309e0c243082 model-00010-of-00029.safetensors
+    fc64ad3efc6a742ea4687c848f9e77f51aeb2f884c2ee5e2fa2184c6bb345fe4 model-00011-of-00029.safetensors
+    93840fcea6c6612de1fac0afbdd2e38c3f294224bfbef4630166f67f3e71553e model-00012-of-00029.safetensors
+    186ab5d446ec26a5db0a320b0259906d05ef2c050dc15d5873d47e850cc92d19 model-00013-of-00029.safetensors
+    4f1a3c8b58ad8e5fead20c5b09943a40384a5c1a5e1e98e982897e94b9a38c2f model-00014-of-00029.safetensors
+    486219f13bb1fa91627ea3e4550b36a3204226f58c935bb56ca6788ed9566964 model-00015-of-00029.safetensors
+    9265de7d0f2b2c4d2e31efbdb68fc70b412e1e869524fc1ac3bf9a9277a5aa5d model-00016-of-00029.safetensors
+    6a3616f3184eea9b5ffedd90d54beb2f93cd3b7df8b5e04c551df86f07dba6dc model-00017-of-00029.safetensors
+    24e9e4b1d36109a5b64376a56c13ca606d576b38f0582b2ea13fb9eeea01faf7 model-00018-of-00029.safetensors
+    93c03602ef7b4c30a0d88c98e48cec344ac013a893ac0de1ff5d9018998b549f model-00019-of-00029.safetensors
+    d7b39b8a54405df776189ef191a9566dda8ed66563d92be152cc2d64ec42e39e model-00020-of-00029.safetensors
+    3ddd8b2c0a61734350c5bd5ab9dae5e11f7f3ea7448276257ff8d0e400ee9d33 model-00021-of-00029.safetensors
+    fb4d35baf1eb157f688f9c083f7eae4ba5b76b0a3800e4fcbe1b65816c283128 model-00022-of-00029.safetensors
+    307b2caac9fc4288c6b204838804fd674ddb4e66d0d7a8559e9c66a915ef77b2 model-00023-of-00029.safetensors
+    4b8df04d70b123304e3077389c455f49fe596f407eff3bf5b0477f7f165192b6 model-00024-of-00029.safetensors
+    a959e57d3fc6ccc6bfebcfa586309729c1adae83258997c5d7a62df103906cd8 model-00025-of-00029.safetensors
+    b7889ee97e1b41a66919370dd9c4149c540cda65201703c6da844a6e79dad6e7 model-00026-of-00029.safetensors
+    f4331698554aed3c32d693fa11b7ad3779483b08e2de4ebad07570fcde94138f model-00027-of-00029.safetensors
+    bbe9e4dd50681e7384271910f5f2be4731b1e908882a2cb20c4178cfc4b10f4e model-00028-of-00029.safetensors
+    e627bf26330ad67479fb8ea824cf88281445ec44751abf7cdbde71044b6f36bd model-00029-of-00029.safetensors
+    b57d733a8e463d314deafbbce631af0ef3a694cacb4dbcacccfd01ac237911ee pytorch_model-00001-of-00029.bin
+    14e5801a430eaa8cd47640a018f175cebbf58d8211edabf7848a308375b28614 pytorch_model-00002-of-00029.bin
+    12700183ce7e3802e0163617835057ca0b4e65e9321d7324d2350db9ecd28663 pytorch_model-00003-of-00029.bin
+    394c6553b66a23e0dc38c6da149704044ef10844fbf15f168a431423f4841eee pytorch_model-00004-of-00029.bin
+    b4aca4a727a3e25e6adef342cb2416fb5ddce962a24e2893ba9462f61ef7068f pytorch_model-00005-of-00029.bin
+    14ac2087bbc50afd3eaa91b053d4c2e9e1495291f21adf35216914e2e7ff8488 pytorch_model-00006-of-00029.bin
+    ebf2b61acd534c3c582bd464e10e2188db07f37f622ba5fdbfcccfa5aafdcaee pytorch_model-00007-of-00029.bin
+    72cee6fbbaf3626ef77a78bfe8da9c63a2dbcd3b462782e1954178ee074d3ed6 pytorch_model-00008-of-00029.bin
+    e483f1e2110505e81de41bc61c7f215c2c789fefa60f8cb6713b398d7fec96a9 pytorch_model-00009-of-00029.bin
+    cb716a00c6d62199bc8ce9b669fa83d473d94c8a51eec1064d249783c7cf8e2a pytorch_model-00010-of-00029.bin
+    c970193eaf1f3af2ea3457d37aeb95cd914326f8fccd70d521d25f6ae4774f1c pytorch_model-00011-of-00029.bin
+    70d14d6a39bbbbbe278973993a5098172343077ac07b5cc389ec6316597820f1 pytorch_model-00012-of-00029.bin
+    c7c2ebef4c61913bba5bfc4fbbe1078f13bded1f74d7fd655ca536fd3284f01d pytorch_model-00013-of-00029.bin
+    54371c04c37f4c933ddae91872eae2903b33d31a025558c11ea963459095130e pytorch_model-00014-of-00029.bin
+    5687b6e63e13553dc8834ffc2f441e99036c76de8866f89f278c281098ba4738 pytorch_model-00015-of-00029.bin
+    1fdf4fc4cc58b5dcd7a656cd94bed58fad59608ab11d62c70536870838a63f66 pytorch_model-00016-of-00029.bin
+    d9a4e08fe6777ea774423cfe7f2ceafd731db86071833b1677cc8ae12666b4e9 pytorch_model-00017-of-00029.bin
+    5bbab57559e01b1729db5b602d165c5944357ca0dfe508e732e3761239358d6f pytorch_model-00018-of-00029.bin
+    317185f76318bfb7338a6d52dfd5124dd488316913a3cdc4725913fb86e4ac61 pytorch_model-00019-of-00029.bin
+    cf585a5974761ae9e95d1da305a56e4689ad01ff6150eb7d467752e3b16d2038 pytorch_model-00020-of-00029.bin
+    6fc99f2ce6bde8b0e0d78d422e96fc52d6d44adf0fc3b23a8abfb14d5f96aaaf pytorch_model-00021-of-00029.bin
+    a9acc8df0ed94084ae3e5cccea68a8503fe93070bb14e6a799bcb6ee853ab610 pytorch_model-00022-of-00029.bin
+    53f5a8c53454b7902aac0d8dc40f1e7664b23e7842c899791404099dffb86b27 pytorch_model-00023-of-00029.bin
+    9cd5032835ed6f02624409e75f60c59df5e8166d7af39430e7a3bb6c42f29721 pytorch_model-00024-of-00029.bin
+    51df08b7ce1f7a83c24249373cbeadee8ec2568a92dd6d4dbb4cdac13012f359 pytorch_model-00025-of-00029.bin
+    587174159f6e03bcdca10a4887839ef390210ae028a7fba7b96da7062ae3b695 pytorch_model-00026-of-00029.bin
+    22f333386db4e1ab7142b90d5a32d4a1e6e3cdd2191f33c2f0cc8f2fa9ca5d54 pytorch_model-00027-of-00029.bin
+    22af339e7cad6e69bdc0eac2b3950f876015b0318815811068ab2dbefc1ffb56 pytorch_model-00028-of-00029.bin
+    cc1fc7dafce8e9511e6470cdf7b81950b2e51dae9b60e0db6a25b5c2faa621cf pytorch_model-00029-of-00029.bin
+    99049b351301fb75b3b0587a484b675cbfd51abe27d2b92eabd385e4c41f97e9 tokenizer.model

model.safetensors.index.json

@@ -0,0 +1,730 @@
+{
+    "metadata": {
+        "total_size": 137953820672
+    },
+    "weight_map": {
+        "lm_head.weight": "model-00029-of-00029.safetensors",
+        "model.embed_tokens.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.input_layernorm.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.mlp.down_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.mlp.up_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.input_layernorm.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.mlp.down_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.mlp.up_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.10.input_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.mlp.down_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.mlp.gate_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.post_attention_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.self_attn.k_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.self_attn.o_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.self_attn.q_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.self_attn.v_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.11.input_layernorm.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.mlp.down_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.mlp.gate_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.mlp.up_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.post_attention_layernorm.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.input_layernorm.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.mlp.down_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.mlp.gate_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.mlp.up_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.post_attention_layernorm.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.input_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.13.mlp.down_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.13.mlp.gate_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.mlp.up_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.post_attention_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.13.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.14.input_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.mlp.down_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.mlp.up_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.post_attention_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.self_attn.k_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.self_attn.o_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.self_attn.q_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.self_attn.v_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.input_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.mlp.down_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.mlp.up_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.post_attention_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.self_attn.k_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.self_attn.o_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.self_attn.q_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.self_attn.v_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.input_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.16.mlp.down_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.16.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.16.post_attention_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.16.self_attn.k_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.self_attn.o_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.self_attn.q_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.self_attn.v_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.17.input_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.mlp.down_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.mlp.gate_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.post_attention_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.self_attn.k_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.self_attn.o_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.self_attn.q_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.self_attn.v_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.input_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.mlp.down_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.mlp.gate_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.post_attention_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.self_attn.k_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.self_attn.o_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.self_attn.q_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.self_attn.v_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.19.input_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.mlp.gate_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.mlp.up_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.self_attn.k_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.19.self_attn.o_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.19.self_attn.q_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.19.self_attn.v_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.2.input_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.2.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.2.mlp.up_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.2.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.20.input_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.mlp.gate_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.mlp.up_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.self_attn.o_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.input_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.mlp.gate_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.mlp.up_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.self_attn.o_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.22.input_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.mlp.down_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.mlp.up_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.post_attention_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.22.self_attn.o_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.22.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.23.input_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.mlp.down_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.mlp.up_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.post_attention_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.self_attn.k_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.self_attn.o_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.self_attn.q_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.self_attn.v_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.input_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.mlp.down_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.mlp.up_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.post_attention_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.self_attn.k_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.self_attn.o_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.self_attn.q_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.self_attn.v_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.25.input_layernorm.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.mlp.down_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.mlp.gate_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.post_attention_layernorm.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.self_attn.k_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.self_attn.o_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.self_attn.q_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.self_attn.v_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.input_layernorm.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.mlp.down_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.mlp.gate_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.post_attention_layernorm.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.self_attn.k_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.self_attn.o_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.self_attn.q_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.self_attn.v_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.input_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.27.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.27.mlp.gate_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.27.self_attn.k_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.self_attn.o_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.self_attn.q_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.self_attn.v_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.28.input_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.mlp.gate_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.mlp.up_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.input_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.mlp.gate_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.mlp.up_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.3.input_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.mlp.gate_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.mlp.up_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.30.input_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.30.mlp.down_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.30.mlp.gate_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.30.mlp.up_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.30.post_attention_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.30.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.30.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.30.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.30.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.31.input_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.mlp.down_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.mlp.gate_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.mlp.up_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.post_attention_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.self_attn.k_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.self_attn.o_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.self_attn.q_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.self_attn.v_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.input_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.mlp.down_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.mlp.gate_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.mlp.up_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.post_attention_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.self_attn.k_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.self_attn.o_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.self_attn.q_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.self_attn.v_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.33.input_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.mlp.down_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.mlp.gate_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.post_attention_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.self_attn.k_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.33.self_attn.o_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.33.self_attn.q_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.33.self_attn.v_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.34.input_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.mlp.down_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.mlp.gate_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.post_attention_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.self_attn.k_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.self_attn.o_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.self_attn.q_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.self_attn.v_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.input_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.mlp.down_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.mlp.gate_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.post_attention_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.self_attn.k_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.self_attn.o_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.self_attn.q_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.self_attn.v_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.36.input_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.mlp.gate_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.mlp.up_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.self_attn.k_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.36.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.self_attn.q_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.36.self_attn.v_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.37.input_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.mlp.gate_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.mlp.up_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.self_attn.k_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.self_attn.q_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.self_attn.v_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.input_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.mlp.gate_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.mlp.up_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.self_attn.k_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.self_attn.q_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.self_attn.v_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.39.input_layernorm.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.mlp.down_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.mlp.up_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.post_attention_layernorm.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.self_attn.k_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.self_attn.o_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.self_attn.q_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.self_attn.v_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.4.input_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.mlp.gate_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.mlp.up_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.40.input_layernorm.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.mlp.down_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.mlp.up_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.post_attention_layernorm.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.self_attn.k_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.self_attn.o_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.self_attn.q_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.self_attn.v_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.input_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.41.mlp.down_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.41.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.mlp.up_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.post_attention_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.41.self_attn.k_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.self_attn.o_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.self_attn.q_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.self_attn.v_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.42.input_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.mlp.down_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.mlp.gate_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.mlp.up_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.post_attention_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.self_attn.k_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.self_attn.o_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.self_attn.q_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.self_attn.v_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.input_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.mlp.down_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.mlp.gate_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.mlp.up_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.post_attention_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.self_attn.k_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.self_attn.o_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.self_attn.q_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.self_attn.v_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.input_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.44.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.44.mlp.gate_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.mlp.up_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.44.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.44.self_attn.k_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.self_attn.o_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.self_attn.q_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.self_attn.v_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.45.input_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.mlp.gate_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.mlp.up_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.input_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.mlp.gate_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.mlp.up_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.47.input_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.mlp.down_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.mlp.up_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.post_attention_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.47.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.47.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.47.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.48.input_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.mlp.down_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.mlp.up_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.post_attention_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.self_attn.k_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.self_attn.o_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.self_attn.q_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.self_attn.v_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.input_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.mlp.down_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.mlp.up_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.post_attention_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.self_attn.k_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.self_attn.o_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.self_attn.q_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.self_attn.v_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.5.input_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.mlp.down_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.mlp.up_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.post_attention_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.50.input_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.mlp.down_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.mlp.gate_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.post_attention_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.self_attn.k_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.50.self_attn.o_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.self_attn.q_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.50.self_attn.v_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.51.input_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.mlp.down_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.mlp.gate_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.post_attention_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.self_attn.k_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.self_attn.o_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.self_attn.q_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.self_attn.v_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.input_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.mlp.down_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.mlp.gate_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.post_attention_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.self_attn.k_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.self_attn.o_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.self_attn.q_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.self_attn.v_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.53.input_layernorm.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.mlp.down_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.mlp.gate_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.mlp.up_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.post_attention_layernorm.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.input_layernorm.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.mlp.down_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.mlp.gate_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.mlp.up_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.post_attention_layernorm.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.input_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.55.mlp.down_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.55.mlp.gate_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.mlp.up_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.post_attention_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.55.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.56.input_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.mlp.down_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.mlp.up_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.post_attention_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.self_attn.k_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.self_attn.o_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.self_attn.q_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.self_attn.v_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.input_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.mlp.down_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.mlp.up_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.post_attention_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.self_attn.k_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.self_attn.o_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.self_attn.q_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.self_attn.v_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.input_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.58.mlp.down_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.58.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.58.post_attention_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.58.self_attn.k_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.self_attn.o_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.self_attn.q_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.self_attn.v_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.59.input_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.mlp.down_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.mlp.gate_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.post_attention_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.self_attn.k_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.self_attn.o_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.self_attn.q_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.self_attn.v_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.6.input_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.mlp.down_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.mlp.up_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.post_attention_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.self_attn.k_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.self_attn.o_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.self_attn.q_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.self_attn.v_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.60.input_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.mlp.down_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.mlp.gate_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.post_attention_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.self_attn.k_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.self_attn.o_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.self_attn.q_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.self_attn.v_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.61.input_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.mlp.gate_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.mlp.up_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.self_attn.k_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.61.self_attn.o_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.61.self_attn.q_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.61.self_attn.v_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.62.input_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.mlp.gate_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.mlp.up_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.self_attn.o_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.input_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.mlp.gate_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.mlp.up_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.self_attn.o_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.64.input_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.mlp.down_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.mlp.up_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.post_attention_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.64.self_attn.o_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.64.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.65.input_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.mlp.down_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.mlp.up_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.post_attention_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.self_attn.k_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.self_attn.o_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.self_attn.q_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.self_attn.v_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.input_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.mlp.down_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.mlp.up_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.post_attention_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.self_attn.k_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.self_attn.o_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.self_attn.q_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.self_attn.v_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.67.input_layernorm.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.mlp.down_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.mlp.gate_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.post_attention_layernorm.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.self_attn.k_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.self_attn.o_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.self_attn.q_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.self_attn.v_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.input_layernorm.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.mlp.down_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.mlp.gate_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.post_attention_layernorm.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.self_attn.k_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.self_attn.o_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.self_attn.q_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.self_attn.v_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.input_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.69.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.69.mlp.gate_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.69.self_attn.k_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.self_attn.o_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.self_attn.q_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.self_attn.v_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.7.input_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.mlp.down_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.mlp.up_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.post_attention_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.self_attn.k_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.self_attn.o_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.self_attn.q_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.self_attn.v_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.70.input_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.mlp.gate_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.mlp.up_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.input_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.mlp.gate_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.mlp.up_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.input_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.72.mlp.down_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.72.mlp.gate_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.mlp.up_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.72.post_attention_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.72.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.73.input_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.mlp.down_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.mlp.gate_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.mlp.up_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.post_attention_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.self_attn.k_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.self_attn.o_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.self_attn.q_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.self_attn.v_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.input_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.mlp.down_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.mlp.gate_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.mlp.up_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.post_attention_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.self_attn.k_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.self_attn.o_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.self_attn.q_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.self_attn.v_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.75.input_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.mlp.down_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.mlp.gate_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.post_attention_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.self_attn.k_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.75.self_attn.o_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.75.self_attn.q_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.75.self_attn.v_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.76.input_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.mlp.down_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.mlp.gate_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.post_attention_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.self_attn.k_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.self_attn.o_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.self_attn.q_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.self_attn.v_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.input_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.mlp.down_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.mlp.gate_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.post_attention_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.self_attn.k_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.self_attn.o_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.self_attn.q_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.self_attn.v_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.78.input_layernorm.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.mlp.down_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.mlp.gate_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.mlp.up_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.post_attention_layernorm.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.self_attn.k_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.78.self_attn.o_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.self_attn.q_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.78.self_attn.v_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.79.input_layernorm.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.mlp.down_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.mlp.gate_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.mlp.up_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.post_attention_layernorm.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.self_attn.k_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.self_attn.o_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.self_attn.q_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.self_attn.v_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.8.input_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.mlp.down_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.mlp.gate_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.post_attention_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.self_attn.k_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.8.self_attn.o_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.self_attn.q_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.8.self_attn.v_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.9.input_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.mlp.down_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.mlp.gate_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.post_attention_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.self_attn.k_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.self_attn.o_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.self_attn.q_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.self_attn.v_proj.weight": "model-00004-of-00029.safetensors",
+        "model.norm.weight": "model-00029-of-00029.safetensors"
+    }
+}

output-00001-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fee02a532048833062197e47d1e13a0dd06262f6b01bbf128503a142585dc76
+size 8552423536

output-00002-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b103483d5aede00f46e848c268263d49b153441f2fe9b31323480f223be1168b
+size 8569624388

output-00003-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67df7e669a6dc4ef23dc46f8adc9cc14e706a82bd65ff492471d4b0db2587bd9
+size 8505826668

output-00004-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35655ac8f2a0be25c19486edd746ebd96dfecfa267f7dffed36603b9e7ccae89
+size 8572902584

output-00005-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6b488d59e598f5eb7bec98c8bcf6501a8ea877665276a15f58ef64553ad500b
+size 8429974572

output-00006-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aefbd80cd99b0257fb7c28caa73f2ec269bf6afb1ee1086fbdc497dea1cb7aa7
+size 908177596

pytorch_model.bin.index.json

@@ -0,0 +1,730 @@
+{
+  "metadata": {
+    "total_size": 137953820672
+  },
+  "weight_map": {
+    "lm_head.weight": "pytorch_model-00029-of-00029.bin",
+    "model.embed_tokens.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.10.input_layernorm.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.10.mlp.down_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.10.mlp.up_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.11.input_layernorm.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.11.mlp.down_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.11.mlp.up_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.input_layernorm.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.mlp.down_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.mlp.up_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.13.input_layernorm.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.13.mlp.down_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.13.mlp.up_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00005-of-00029.bin",
+    "model.layers.14.input_layernorm.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.14.mlp.down_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.14.mlp.up_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.input_layernorm.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.mlp.down_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.mlp.up_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.16.input_layernorm.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.16.mlp.down_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.16.mlp.up_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00006-of-00029.bin",
+    "model.layers.17.input_layernorm.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.17.mlp.down_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.17.mlp.up_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.input_layernorm.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.mlp.down_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.mlp.up_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.19.input_layernorm.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.19.mlp.down_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.19.mlp.up_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00007-of-00029.bin",
+    "model.layers.2.input_layernorm.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.2.mlp.down_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.2.mlp.up_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00029.bin",
+    "model.layers.20.input_layernorm.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.20.mlp.down_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.20.mlp.up_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.input_layernorm.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.mlp.down_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.mlp.up_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.22.input_layernorm.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.22.mlp.down_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.22.mlp.up_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00008-of-00029.bin",
+    "model.layers.23.input_layernorm.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.23.mlp.down_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.23.mlp.up_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.input_layernorm.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.mlp.down_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.mlp.up_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00009-of-00029.bin",
+    "model.layers.25.input_layernorm.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.25.mlp.down_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.25.mlp.up_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.input_layernorm.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.mlp.down_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.mlp.up_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.27.input_layernorm.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.27.mlp.down_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.27.mlp.up_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00010-of-00029.bin",
+    "model.layers.28.input_layernorm.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.28.mlp.down_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.28.mlp.up_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.input_layernorm.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.mlp.down_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.mlp.up_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.3.input_layernorm.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.3.mlp.down_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.3.mlp.up_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.30.input_layernorm.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.30.mlp.down_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.30.mlp.up_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00011-of-00029.bin",
+    "model.layers.31.input_layernorm.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.31.mlp.down_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.31.mlp.up_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.input_layernorm.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.mlp.down_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.mlp.gate_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.mlp.up_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.post_attention_layernorm.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.self_attn.k_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.self_attn.o_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.self_attn.q_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.32.self_attn.v_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.33.input_layernorm.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.33.mlp.down_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.33.mlp.gate_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.33.mlp.up_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.33.post_attention_layernorm.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.33.self_attn.k_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.33.self_attn.o_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.33.self_attn.q_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.33.self_attn.v_proj.weight": "pytorch_model-00012-of-00029.bin",
+    "model.layers.34.input_layernorm.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.34.mlp.down_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.34.mlp.gate_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.34.mlp.up_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.34.post_attention_layernorm.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.34.self_attn.k_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.34.self_attn.o_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.34.self_attn.q_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.34.self_attn.v_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.input_layernorm.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.mlp.down_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.mlp.gate_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.mlp.up_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.post_attention_layernorm.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.self_attn.k_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.self_attn.o_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.self_attn.q_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.35.self_attn.v_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.36.input_layernorm.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.36.mlp.down_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.36.mlp.gate_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.36.mlp.up_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.36.post_attention_layernorm.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.36.self_attn.k_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.36.self_attn.o_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.36.self_attn.q_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.36.self_attn.v_proj.weight": "pytorch_model-00013-of-00029.bin",
+    "model.layers.37.input_layernorm.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.37.mlp.down_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.37.mlp.gate_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.37.mlp.up_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.37.post_attention_layernorm.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.37.self_attn.k_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.37.self_attn.o_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.37.self_attn.q_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.37.self_attn.v_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.input_layernorm.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.mlp.down_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.mlp.gate_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.mlp.up_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.post_attention_layernorm.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.self_attn.k_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.self_attn.o_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.self_attn.q_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.38.self_attn.v_proj.weight": "pytorch_model-00014-of-00029.bin",
+    "model.layers.39.input_layernorm.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.39.mlp.down_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.39.mlp.gate_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.39.mlp.up_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.39.post_attention_layernorm.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.39.self_attn.k_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.39.self_attn.o_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.39.self_attn.q_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.39.self_attn.v_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.4.input_layernorm.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.4.mlp.down_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.4.mlp.up_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.40.input_layernorm.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.40.mlp.down_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.40.mlp.gate_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.40.mlp.up_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.40.post_attention_layernorm.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.40.self_attn.k_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.40.self_attn.o_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.40.self_attn.q_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.40.self_attn.v_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.41.input_layernorm.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.41.mlp.down_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.41.mlp.gate_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.41.mlp.up_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.41.post_attention_layernorm.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.41.self_attn.k_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.41.self_attn.o_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.41.self_attn.q_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.41.self_attn.v_proj.weight": "pytorch_model-00015-of-00029.bin",
+    "model.layers.42.input_layernorm.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.42.mlp.down_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.42.mlp.gate_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.42.mlp.up_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.42.post_attention_layernorm.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.42.self_attn.k_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.42.self_attn.o_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.42.self_attn.q_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.42.self_attn.v_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.input_layernorm.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.mlp.down_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.mlp.gate_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.mlp.up_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.post_attention_layernorm.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.self_attn.k_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.self_attn.o_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.self_attn.q_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.43.self_attn.v_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.44.input_layernorm.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.44.mlp.down_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.44.mlp.gate_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.44.mlp.up_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.44.post_attention_layernorm.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.44.self_attn.k_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.44.self_attn.o_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.44.self_attn.q_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.44.self_attn.v_proj.weight": "pytorch_model-00016-of-00029.bin",
+    "model.layers.45.input_layernorm.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.45.mlp.down_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.45.mlp.gate_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.45.mlp.up_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.45.post_attention_layernorm.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.45.self_attn.k_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.45.self_attn.o_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.45.self_attn.q_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.45.self_attn.v_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.input_layernorm.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.mlp.down_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.mlp.gate_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.mlp.up_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.post_attention_layernorm.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.self_attn.k_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.self_attn.o_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.self_attn.q_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.46.self_attn.v_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.47.input_layernorm.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.47.mlp.down_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.47.mlp.gate_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.47.mlp.up_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.47.post_attention_layernorm.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.47.self_attn.k_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.47.self_attn.o_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.47.self_attn.q_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.47.self_attn.v_proj.weight": "pytorch_model-00017-of-00029.bin",
+    "model.layers.48.input_layernorm.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.48.mlp.down_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.48.mlp.gate_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.48.mlp.up_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.48.post_attention_layernorm.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.48.self_attn.k_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.48.self_attn.o_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.48.self_attn.q_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.48.self_attn.v_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.input_layernorm.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.mlp.down_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.mlp.gate_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.mlp.up_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.post_attention_layernorm.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.self_attn.k_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.self_attn.o_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.self_attn.q_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.49.self_attn.v_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.5.input_layernorm.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.5.mlp.down_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.5.mlp.up_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00002-of-00029.bin",
+    "model.layers.50.input_layernorm.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.50.mlp.down_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.50.mlp.gate_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.50.mlp.up_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.50.post_attention_layernorm.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.50.self_attn.k_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.50.self_attn.o_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.50.self_attn.q_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.50.self_attn.v_proj.weight": "pytorch_model-00018-of-00029.bin",
+    "model.layers.51.input_layernorm.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.51.mlp.down_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.51.mlp.gate_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.51.mlp.up_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.51.post_attention_layernorm.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.51.self_attn.k_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.51.self_attn.o_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.51.self_attn.q_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.51.self_attn.v_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.input_layernorm.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.mlp.down_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.mlp.gate_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.mlp.up_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.post_attention_layernorm.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.self_attn.k_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.self_attn.o_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.self_attn.q_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.52.self_attn.v_proj.weight": "pytorch_model-00019-of-00029.bin",
+    "model.layers.53.input_layernorm.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.53.mlp.down_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.53.mlp.gate_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.53.mlp.up_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.53.post_attention_layernorm.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.53.self_attn.k_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.53.self_attn.o_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.53.self_attn.q_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.53.self_attn.v_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.input_layernorm.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.mlp.down_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.mlp.gate_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.mlp.up_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.post_attention_layernorm.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.self_attn.k_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.self_attn.o_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.self_attn.q_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.54.self_attn.v_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.55.input_layernorm.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.55.mlp.down_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.55.mlp.gate_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.55.mlp.up_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.55.post_attention_layernorm.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.55.self_attn.k_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.55.self_attn.o_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.55.self_attn.q_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.55.self_attn.v_proj.weight": "pytorch_model-00020-of-00029.bin",
+    "model.layers.56.input_layernorm.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.56.mlp.down_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.56.mlp.gate_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.56.mlp.up_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.56.post_attention_layernorm.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.56.self_attn.k_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.56.self_attn.o_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.56.self_attn.q_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.56.self_attn.v_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.input_layernorm.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.mlp.down_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.mlp.gate_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.mlp.up_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.post_attention_layernorm.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.self_attn.k_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.self_attn.o_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.self_attn.q_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.57.self_attn.v_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.58.input_layernorm.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.58.mlp.down_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.58.mlp.gate_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.58.mlp.up_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.58.post_attention_layernorm.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.58.self_attn.k_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.58.self_attn.o_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.58.self_attn.q_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.58.self_attn.v_proj.weight": "pytorch_model-00021-of-00029.bin",
+    "model.layers.59.input_layernorm.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.59.mlp.down_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.59.mlp.gate_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.59.mlp.up_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.59.post_attention_layernorm.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.59.self_attn.k_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.59.self_attn.o_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.59.self_attn.q_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.59.self_attn.v_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.6.input_layernorm.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.6.mlp.down_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.6.mlp.up_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.60.input_layernorm.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.60.mlp.down_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.60.mlp.gate_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.60.mlp.up_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.60.post_attention_layernorm.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.60.self_attn.k_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.60.self_attn.o_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.60.self_attn.q_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.60.self_attn.v_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.61.input_layernorm.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.61.mlp.down_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.61.mlp.gate_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.61.mlp.up_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.61.post_attention_layernorm.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.61.self_attn.k_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.61.self_attn.o_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.61.self_attn.q_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.61.self_attn.v_proj.weight": "pytorch_model-00022-of-00029.bin",
+    "model.layers.62.input_layernorm.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.62.mlp.down_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.62.mlp.gate_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.62.mlp.up_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.62.post_attention_layernorm.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.62.self_attn.k_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.62.self_attn.o_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.62.self_attn.q_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.62.self_attn.v_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.input_layernorm.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.mlp.down_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.mlp.gate_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.mlp.up_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.post_attention_layernorm.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.self_attn.k_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.self_attn.o_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.self_attn.q_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.63.self_attn.v_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.64.input_layernorm.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.64.mlp.down_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.64.mlp.gate_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.64.mlp.up_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.64.post_attention_layernorm.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.64.self_attn.k_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.64.self_attn.o_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.64.self_attn.q_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.64.self_attn.v_proj.weight": "pytorch_model-00023-of-00029.bin",
+    "model.layers.65.input_layernorm.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.65.mlp.down_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.65.mlp.gate_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.65.mlp.up_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.65.post_attention_layernorm.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.65.self_attn.k_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.65.self_attn.o_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.65.self_attn.q_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.65.self_attn.v_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.input_layernorm.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.mlp.down_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.mlp.gate_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.mlp.up_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.post_attention_layernorm.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.self_attn.k_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.self_attn.o_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.self_attn.q_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.66.self_attn.v_proj.weight": "pytorch_model-00024-of-00029.bin",
+    "model.layers.67.input_layernorm.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.67.mlp.down_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.67.mlp.gate_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.67.mlp.up_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.67.post_attention_layernorm.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.67.self_attn.k_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.67.self_attn.o_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.67.self_attn.q_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.67.self_attn.v_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.input_layernorm.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.mlp.down_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.mlp.gate_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.mlp.up_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.post_attention_layernorm.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.self_attn.k_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.self_attn.o_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.self_attn.q_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.68.self_attn.v_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.69.input_layernorm.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.69.mlp.down_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.69.mlp.gate_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.69.mlp.up_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.69.post_attention_layernorm.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.69.self_attn.k_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.69.self_attn.o_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.69.self_attn.q_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.69.self_attn.v_proj.weight": "pytorch_model-00025-of-00029.bin",
+    "model.layers.7.input_layernorm.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.7.mlp.down_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.7.mlp.up_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.70.input_layernorm.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.70.mlp.down_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.70.mlp.gate_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.70.mlp.up_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.70.post_attention_layernorm.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.70.self_attn.k_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.70.self_attn.o_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.70.self_attn.q_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.70.self_attn.v_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.input_layernorm.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.mlp.down_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.mlp.gate_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.mlp.up_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.post_attention_layernorm.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.self_attn.k_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.self_attn.o_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.self_attn.q_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.71.self_attn.v_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.72.input_layernorm.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.72.mlp.down_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.72.mlp.gate_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.72.mlp.up_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.72.post_attention_layernorm.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.72.self_attn.k_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.72.self_attn.o_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.72.self_attn.q_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.72.self_attn.v_proj.weight": "pytorch_model-00026-of-00029.bin",
+    "model.layers.73.input_layernorm.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.73.mlp.down_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.73.mlp.gate_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.73.mlp.up_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.73.post_attention_layernorm.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.73.self_attn.k_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.73.self_attn.o_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.73.self_attn.q_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.73.self_attn.v_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.input_layernorm.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.mlp.down_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.mlp.gate_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.mlp.up_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.post_attention_layernorm.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.self_attn.k_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.self_attn.o_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.self_attn.q_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.74.self_attn.v_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.75.input_layernorm.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.75.mlp.down_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.75.mlp.gate_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.75.mlp.up_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.75.post_attention_layernorm.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.75.self_attn.k_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.75.self_attn.o_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.75.self_attn.q_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.75.self_attn.v_proj.weight": "pytorch_model-00027-of-00029.bin",
+    "model.layers.76.input_layernorm.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.76.mlp.down_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.76.mlp.gate_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.76.mlp.up_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.76.post_attention_layernorm.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.76.self_attn.k_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.76.self_attn.o_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.76.self_attn.q_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.76.self_attn.v_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.input_layernorm.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.mlp.down_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.mlp.gate_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.mlp.up_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.post_attention_layernorm.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.self_attn.k_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.self_attn.o_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.self_attn.q_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.77.self_attn.v_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.78.input_layernorm.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.78.mlp.down_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.78.mlp.gate_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.78.mlp.up_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.78.post_attention_layernorm.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.78.self_attn.k_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.78.self_attn.o_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.78.self_attn.q_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.78.self_attn.v_proj.weight": "pytorch_model-00028-of-00029.bin",
+    "model.layers.79.input_layernorm.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.79.mlp.down_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.79.mlp.gate_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.79.mlp.up_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.79.post_attention_layernorm.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.79.self_attn.k_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.79.self_attn.o_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.79.self_attn.q_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.79.self_attn.v_proj.weight": "pytorch_model-00029-of-00029.bin",
+    "model.layers.8.input_layernorm.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.8.mlp.down_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.8.mlp.up_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00003-of-00029.bin",
+    "model.layers.9.input_layernorm.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.9.mlp.down_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.9.mlp.up_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00004-of-00029.bin",
+    "model.norm.weight": "pytorch_model-00029-of-00029.bin"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,23 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99049b351301fb75b3b0587a484b675cbfd51abe27d2b92eabd385e4c41f97e9
+size 500033

tokenizer_config.json

@@ -0,0 +1,41 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": null,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}