7cdcbda0b590b6317c3268cb1c649f9626ca7cf3ffd0703bb5265adf68984594

README.md

@@ -0,0 +1,85 @@
+---
+thumbnail: "https://assets-global.website-files.com/646b351987a8d8ce158d1940/64ec9e96b4334c0e1ac41504_Logo%20with%20white%20text.svg"
+base_model: Qwen/Qwen2-7B-Instruct
+metrics:
+- memory_disk
+- memory_inference
+- inference_latency
+- inference_throughput
+- inference_CO2_emissions
+- inference_energy_consumption
+tags:
+- pruna-ai
+---
+<!-- header start -->
+<!-- 200823 -->
+<div style="width: auto; margin-left: auto; margin-right: auto">
+    <a href="https://www.pruna.ai/" target="_blank" rel="noopener noreferrer">
+        <img src="https://i.imgur.com/eDAlcgk.png" alt="PrunaAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+    </a>
+</div>
+<!-- header end -->
+
+[![Twitter](https://img.shields.io/twitter/follow/PrunaAI?style=social)](https://twitter.com/PrunaAI)
+[![GitHub](https://img.shields.io/github/followers/PrunaAI?label=Follow%20%40PrunaAI&style=social)](https://github.com/PrunaAI)
+[![LinkedIn](https://img.shields.io/badge/LinkedIn-Connect-blue)](https://www.linkedin.com/company/93832878/admin/feed/posts/?feedType=following)
+[![Discord](https://img.shields.io/badge/Discord-Join%20Us-blue?style=social&logo=discord)](https://discord.gg/rskEr4BZJx)
+
+# Simply make AI models cheaper, smaller, faster, and greener!
+
+- Give a thumbs up if you like this model!
+- Contact us and tell us which model to compress next [here](https://www.pruna.ai/contact).
+- Request access to easily compress your *own* AI models [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
+- Read the documentations to know more [here](https://pruna-ai-pruna.readthedocs-hosted.com/en/latest/)
+- Join Pruna AI community on Discord [here](https://discord.gg/CP4VSgck) to share feedback/suggestions or get help.
+
+## Results
+
+![image info](./plots.png)
+
+**Frequently Asked Questions**
+- ***How does the compression work?*** The model is compressed with llm-int8.
+- ***How does the model quality change?*** The quality of the model output might vary compared to the base model.
+- ***How is the model efficiency evaluated?*** These results were obtained on HARDWARE_NAME with configuration described in `model/smash_config.json` and are obtained after a hardware warmup. The smashed model is directly compared to the original base model. Efficiency results may vary in other settings (e.g. other hardware, image size, batch size, ...). We recommend to directly run them in the use-case conditions to know if the smashed model can benefit you.
+- ***What is the model format?*** We use safetensors.
+- ***What calibration data has been used?*** If needed by the compression method, we used WikiText as the calibration data.
+- ***What is the naming convention for Pruna Huggingface models?*** We take the original model name and append "turbo", "tiny", or "green" if the smashed model has a measured inference speed, inference memory, or inference energy consumption which is less than 90% of the original base model.
+- ***How to compress my own models?*** You can request premium access to more compression methods and tech support for your specific use-cases [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
+- ***What are "first" metrics?*** Results mentioning "first" are obtained after the first run of the model. The first run might take more memory or be slower than the subsequent runs due cuda overheads.
+- ***What are "Sync" and "Async" metrics?*** "Sync" metrics are obtained by syncing all GPU processes and stop measurement when all of them are executed. "Async" metrics are obtained without syncing all GPU processes and stop when the model output can be used by the CPU. We provide both metrics since both could be relevant depending on the use-case. We recommend to test the efficiency gains directly in your use-cases.
+
+## Setup
+
+You can run the smashed model with these steps:
+
+0. Check requirements from the original repo Qwen/Qwen2-7B-Instruct installed. In particular, check python, cuda, and transformers versions.
+1. Make sure that you have installed quantization related packages.
+    ```bash
+    pip install transformers accelerate bitsandbytes>0.37.0
+    ```
+2. Load & run the model.
+    ```python 
+   from transformers import AutoModelForCausalLM, AutoTokenizer
+   
+
+   model = AutoModelForCausalLM.from_pretrained("PrunaAI/Qwen-Qwen2-7B-Instruct-bnb-4bit-smashed", trust_remote_code=True, device_map='auto')
+   tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-7B-Instruct")
+    
+   input_ids = tokenizer("What is the color of prunes?,", return_tensors='pt').to(model.device)["input_ids"]
+    
+   outputs = model.generate(input_ids, max_new_tokens=216)
+   tokenizer.decode(outputs[0])
+    ```
+
+## Configurations
+
+The configuration info are in `smash_config.json`.
+
+## Credits & License
+
+The license of the smashed model follows the license of the original model. Please check the license of the original model Qwen/Qwen2-7B-Instruct before using this model which provided the base model. The license  of the `pruna-engine` is [here](https://pypi.org/project/pruna-engine/) on Pypi.
+
+## Want to compress other models?
+
+- Contact us and tell us which model to compress next [here](https://www.pruna.ai/contact).
+- Request access to easily compress your own AI models [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "/ceph/hdd/staff/charpent/.cache/modelsphds6aqhsbpdttln",
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 3584,
+  "initializer_range": 0.02,
+  "intermediate_size": 18944,
+  "max_position_embeddings": 32768,
+  "max_window_layers": 28,
+  "model_type": "qwen2",
+  "num_attention_heads": 28,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 4,
+  "quantization_config": {
+    "_load_in_4bit": true,
+    "_load_in_8bit": false,
+    "bnb_4bit_compute_dtype": "bfloat16",
+    "bnb_4bit_quant_storage": "uint8",
+    "bnb_4bit_quant_type": "fp4",
+    "bnb_4bit_use_double_quant": false,
+    "llm_int8_enable_fp32_cpu_offload": false,
+    "llm_int8_has_fp16_weight": false,
+    "llm_int8_skip_modules": [
+      "lm_head"
+    ],
+    "llm_int8_threshold": 6.0,
+    "load_in_4bit": true,
+    "load_in_8bit": false,
+    "quant_method": "bitsandbytes"
+  },
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 1000000.0,
+  "sliding_window": 131072,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.2",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 152064
+}

generation_config.json

@@ -0,0 +1,14 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "repetition_penalty": 1.05,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "4.41.2"
+}

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38b6eef54ae6fe69f5715367e066a3f8a1c6d233fed8feb5bb358bf107d54660
+size 1089994880

model.safetensors.index.json

@@ -0,0 +1,934 @@
+{
+  "metadata": {
+    "total_size": 5851159328
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00002-of-00002.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.24.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.25.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.26.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.27.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.down_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.gate_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.gate_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.gate_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.up_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight.quant_state.bitsandbytes__fp4": "model-00001-of-00002.safetensors",
+    "model.norm.weight": "model-00001-of-00002.safetensors"
+  }
+}

smash_config.json

@@ -0,0 +1,31 @@
+{
+    "api_key": null,
+    "verify_url": "http://johnrachwan.pythonanywhere.com",
+    "smash_config": {
+        "pruners": "None",
+        "pruning_ratio": 0.0,
+        "factorizers": "None",
+        "quantizers": "['llm-int8']",
+        "weight_quantization_bits": 4,
+        "output_deviation": 0.005,
+        "compilers": "None",
+        "static_batch": true,
+        "static_shape": true,
+        "controlnet": "None",
+        "unet_dim": 4,
+        "device": "cuda",
+        "cache_dir": "/ceph/hdd/staff/charpent/.cache/modelsphds6aqh",
+        "batch_size": 1,
+        "model_name": "Qwen/Qwen2-7B-Instruct",
+        "task": "text_text_generation",
+        "max_batch_size": 1,
+        "qtype_weight": "torch.qint8",
+        "qtype_activation": "torch.quint8",
+        "qobserver": "<class 'torch.ao.quantization.observer.MinMaxObserver'>",
+        "qscheme": "torch.per_tensor_symmetric",
+        "qconfig": "x86",
+        "group_size": 128,
+        "damp_percent": 0.1,
+        "save_load_fn": "bitsandbytes"
+    }
+}

special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "legacy": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}