/Users/cfruan/miniconda3/envs/mlc-chat-venv/bin/python -m mlc_chat gen_config /Users/Shared/models/CodeLlama-70b-Python-hf --quantization q4f32_1 --conv-template codellama_instruct --output /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq --context-window-size 16384
[2024-01-30 23:22:54] INFO auto_config.py:115: Found model configuration: /Users/Shared/models/CodeLlama-70b-Python-hf/config.json
[2024-01-30 23:22:54] INFO auto_config.py:153: Found model type: llama. Use `--model-type` to override.
[2024-01-30 23:22:54] INFO llama_model.py:51: context_window_size not found in config.json. Falling back to max_position_embeddings (4096)
[2024-01-30 23:22:54] INFO llama_model.py:71: prefill_chunk_size defaults to context_window_size (4096)
[2024-01-30 23:22:54] INFO config.py:106: Overriding context_window_size from 4096 to 16384
[2024-01-30 23:22:54] INFO config.py:106: Overriding max_batch_size from 1 to 80
[2024-01-30 23:22:54] INFO gen_config.py:116: [generation_config.json] Setting bos_token_id: 1
[2024-01-30 23:22:54] INFO gen_config.py:116: [generation_config.json] Setting eos_token_id: 2
[2024-01-30 23:22:54] INFO gen_config.py:128: Found tokenizer config: /Users/Shared/models/CodeLlama-70b-Python-hf/tokenizer.model. Copying to /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq/tokenizer.model
[2024-01-30 23:22:54] INFO gen_config.py:128: Found tokenizer config: /Users/Shared/models/CodeLlama-70b-Python-hf/tokenizer.json. Copying to /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq/tokenizer.json
[2024-01-30 23:22:54] INFO gen_config.py:130: Not found tokenizer config: /Users/Shared/models/CodeLlama-70b-Python-hf/vocab.json
[2024-01-30 23:22:54] INFO gen_config.py:130: Not found tokenizer config: /Users/Shared/models/CodeLlama-70b-Python-hf/merges.txt
[2024-01-30 23:22:54] INFO gen_config.py:130: Not found tokenizer config: /Users/Shared/models/CodeLlama-70b-Python-hf/added_tokens.json
[2024-01-30 23:22:54] INFO gen_config.py:128: Found tokenizer config: /Users/Shared/models/CodeLlama-70b-Python-hf/tokenizer_config.json. Copying to /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq/tokenizer_config.json
[2024-01-30 23:22:54] INFO gen_config.py:69: [System default] Setting pad_token_id: 0
[2024-01-30 23:22:54] INFO gen_config.py:69: [System default] Setting temperature: 0.7
[2024-01-30 23:22:54] INFO gen_config.py:69: [System default] Setting repetition_penalty: 1.0
[2024-01-30 23:22:54] INFO gen_config.py:69: [System default] Setting top_p: 0.95
[2024-01-30 23:22:54] INFO gen_config.py:69: [System default] Setting mean_gen_len: 128
[2024-01-30 23:22:54] INFO gen_config.py:69: [System default] Setting max_gen_len: 512
[2024-01-30 23:22:54] INFO gen_config.py:69: [System default] Setting shift_fill_factor: 0.3
[2024-01-30 23:22:54] INFO gen_config.py:158: Dumping configuration file to: /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq/mlc-chat-config.json
/Users/cfruan/miniconda3/envs/mlc-chat-venv/bin/python -m mlc_chat convert_weight /Users/Shared/models/CodeLlama-70b-Python-hf --quantization q4f32_1 --source-format auto --output /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq
[2024-01-30 23:22:54] INFO auto_config.py:115: Found model configuration: /Users/Shared/models/CodeLlama-70b-Python-hf/config.json
[2024-01-30 23:22:55] INFO auto_device.py:85: Not found device: cuda:0
[2024-01-30 23:22:55] INFO auto_device.py:85: Not found device: rocm:0
[2024-01-30 23:22:55] INFO auto_device.py:76: Found device: metal:0
[2024-01-30 23:22:56] INFO auto_device.py:85: Not found device: vulkan:0
[2024-01-30 23:22:56] INFO auto_device.py:85: Not found device: opencl:0
[2024-01-30 23:22:56] INFO auto_device.py:33: Using device: metal:0
[2024-01-30 23:22:56] INFO auto_weight.py:70: Finding weights in: /Users/Shared/models/CodeLlama-70b-Python-hf
[2024-01-30 23:22:56] INFO auto_weight.py:120: Found source weight format: huggingface-torch. Source configuration: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model.bin.index.json
[2024-01-30 23:22:56] INFO auto_weight.py:143: Found source weight format: huggingface-safetensor. Source configuration: /Users/Shared/models/CodeLlama-70b-Python-hf/model.safetensors.index.json
[2024-01-30 23:22:56] INFO auto_weight.py:106: Using source weight configuration: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model.bin.index.json. Use `--source` to override.
[2024-01-30 23:22:56] INFO auto_weight.py:110: Using source weight format: huggingface-torch. Use `--source-format` to override.
[2024-01-30 23:22:56] INFO auto_config.py:153: Found model type: llama. Use `--model-type` to override.
[2024-01-30 23:22:56] INFO llama_model.py:51: context_window_size not found in config.json. Falling back to max_position_embeddings (4096)
[2024-01-30 23:22:56] INFO llama_model.py:71: prefill_chunk_size defaults to context_window_size (4096)
Weight conversion with arguments:
  --config          /Users/Shared/models/CodeLlama-70b-Python-hf/config.json
  --quantization    GroupQuantize(name='q4f32_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float32', linear_weight_layout='NK', num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7)
  --model-type      llama
  --device          metal:0
  --source          /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model.bin.index.json
  --source-format   huggingface-torch
  --output          /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq

  0%|                                                                                                            | 0/483 [00:00<?, ?it/s]
                                                                                                                                         
[2024-01-30 23:23:04] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00029-of-00029.bin

  0%|                                                                                                            | 0/483 [00:00<?, ?it/s]
                                                                                                                                         
[2024-01-30 23:23:06] INFO group_quantization.py:227: Compiling quantize function for key: ((32016, 8192), float32, metal, axis=1, output_transpose=False)

  0%|                                                                                                            | 0/483 [00:02<?, ?it/s]
                                                                                                                                         
[2024-01-30 23:23:06] INFO huggingface_loader.py:121: [Quantized] Parameter: "lm_head.q_weight", shape: (32016, 1024), dtype: uint32

  0%|                                                                                                            | 0/483 [00:02<?, ?it/s]
                                                                                                                                         
[2024-01-30 23:23:06] INFO huggingface_loader.py:121: [Quantized] Parameter: "lm_head.q_scale", shape: (32016, 256), dtype: float32

  0%|                                                                                                            | 0/483 [00:02<?, ?it/s]
  0%|▏                                                                                                   | 1/483 [00:02<20:42,  2.58s/it]
                                                                                                                                         
[2024-01-30 23:23:07] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.78.input_layernorm.weight", shape: (8192,), dtype: float32

  0%|▏                                                                                                   | 1/483 [00:02<20:42,  2.58s/it]
                                                                                                                                         
[2024-01-30 23:23:07] INFO group_quantization.py:227: Compiling quantize function for key: ((8192, 28672), float32, metal, axis=1, output_transpose=False)

  0%|▏                                                                                                   | 1/483 [00:02<20:42,  2.58s/it]
                                                                                                                                         
[2024-01-30 23:23:07] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.78.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

  0%|▏                                                                                                   | 1/483 [00:02<20:42,  2.58s/it]
                                                                                                                                         
[2024-01-30 23:23:07] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.78.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

  0%|▏                                                                                                   | 1/483 [00:02<20:42,  2.58s/it]
  1%|▌                                                                                                   | 3/483 [00:02<06:15,  1.28it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO group_quantization.py:227: Compiling quantize function for key: ((57344, 8192), float32, metal, axis=1, output_transpose=False)

  1%|▌                                                                                                   | 3/483 [00:03<06:15,  1.28it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.78.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

  1%|▌                                                                                                   | 3/483 [00:03<06:15,  1.28it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.78.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

  1%|▌                                                                                                   | 3/483 [00:03<06:15,  1.28it/s]
  1%|▊                                                                                                   | 4/483 [00:03<06:35,  1.21it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.78.post_attention_layernorm.weight", shape: (8192,), dtype: float32

  1%|▊                                                                                                   | 4/483 [00:03<06:35,  1.21it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO group_quantization.py:227: Compiling quantize function for key: ((8192, 8192), float32, metal, axis=1, output_transpose=False)

  1%|▊                                                                                                   | 4/483 [00:03<06:35,  1.21it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.78.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

  1%|▊                                                                                                   | 4/483 [00:03<06:35,  1.21it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.78.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

  1%|▊                                                                                                   | 4/483 [00:03<06:35,  1.21it/s]
  1%|█▏                                                                                                  | 6/483 [00:03<03:36,  2.21it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.79.input_layernorm.weight", shape: (8192,), dtype: float32

  1%|█▏                                                                                                  | 6/483 [00:03<03:36,  2.21it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.79.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

  1%|█▏                                                                                                  | 6/483 [00:04<03:36,  2.21it/s]
                                                                                                                                         
[2024-01-30 23:23:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.79.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

  1%|█▏                                                                                                  | 6/483 [00:04<03:36,  2.21it/s]
  2%|█▋                                                                                                  | 8/483 [00:04<02:32,  3.11it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.79.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

  2%|█▋                                                                                                  | 8/483 [00:04<02:32,  3.11it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.79.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

  2%|█▋                                                                                                  | 8/483 [00:04<02:32,  3.11it/s]
  2%|█▊                                                                                                  | 9/483 [00:04<03:17,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.79.post_attention_layernorm.weight", shape: (8192,), dtype: float32

  2%|█▊                                                                                                  | 9/483 [00:04<03:17,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO group_quantization.py:227: Compiling quantize function for key: ((10240, 8192), float32, metal, axis=1, output_transpose=False)

  2%|█▊                                                                                                  | 9/483 [00:05<03:17,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.79.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

  2%|█▊                                                                                                  | 9/483 [00:05<03:17,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.79.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

  2%|█▊                                                                                                  | 9/483 [00:05<03:17,  2.40it/s]
  2%|██▎                                                                                                | 11/483 [00:05<02:16,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.79.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

  2%|██▎                                                                                                | 11/483 [00:05<02:16,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.79.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

  2%|██▎                                                                                                | 11/483 [00:05<02:16,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.norm.weight", shape: (8192,), dtype: float32

  2%|██▎                                                                                                | 11/483 [00:05<02:16,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00029-of-00029.bin

  2%|██▎                                                                                                | 11/483 [00:05<02:16,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:09] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00001-of-00029.bin

  2%|██▎                                                                                                | 11/483 [00:05<02:16,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:11] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.embed_tokens.q_weight", shape: (32016, 1024), dtype: uint32

  2%|██▎                                                                                                | 11/483 [00:07<02:16,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:11] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.embed_tokens.q_scale", shape: (32016, 256), dtype: float32

  2%|██▎                                                                                                | 11/483 [00:07<02:16,  3.46it/s]
  3%|██▊                                                                                                | 14/483 [00:07<03:31,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:23:11] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.0.input_layernorm.weight", shape: (8192,), dtype: float32

  3%|██▊                                                                                                | 14/483 [00:07<03:31,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:23:11] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.0.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

  3%|██▊                                                                                                | 14/483 [00:07<03:31,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:23:11] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.0.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

  3%|██▊                                                                                                | 14/483 [00:07<03:31,  2.22it/s]
  3%|███▎                                                                                               | 16/483 [00:07<02:40,  2.91it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.0.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

  3%|███▎                                                                                               | 16/483 [00:07<02:40,  2.91it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.0.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

  3%|███▎                                                                                               | 16/483 [00:07<02:40,  2.91it/s]
  4%|███▍                                                                                               | 17/483 [00:07<03:03,  2.53it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.0.post_attention_layernorm.weight", shape: (8192,), dtype: float32

  4%|███▍                                                                                               | 17/483 [00:07<03:03,  2.53it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.0.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

  4%|███▍                                                                                               | 17/483 [00:08<03:03,  2.53it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.0.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

  4%|███▍                                                                                               | 17/483 [00:08<03:03,  2.53it/s]
  4%|███▉                                                                                               | 19/483 [00:08<02:12,  3.51it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.0.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

  4%|███▉                                                                                               | 19/483 [00:08<02:12,  3.51it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.0.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

  4%|███▉                                                                                               | 19/483 [00:08<02:12,  3.51it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.1.input_layernorm.weight", shape: (8192,), dtype: float32

  4%|███▉                                                                                               | 19/483 [00:08<02:12,  3.51it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.1.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

  4%|███▉                                                                                               | 19/483 [00:08<02:12,  3.51it/s]
                                                                                                                                         
[2024-01-30 23:23:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.1.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

  4%|███▉                                                                                               | 19/483 [00:08<02:12,  3.51it/s]
  5%|████▌                                                                                              | 22/483 [00:08<01:34,  4.86it/s]
                                                                                                                                         
[2024-01-30 23:23:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.1.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

  5%|████▌                                                                                              | 22/483 [00:08<01:34,  4.86it/s]
                                                                                                                                         
[2024-01-30 23:23:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.1.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

  5%|████▌                                                                                              | 22/483 [00:08<01:34,  4.86it/s]
  5%|████▋                                                                                              | 23/483 [00:09<02:10,  3.51it/s]
                                                                                                                                         
[2024-01-30 23:23:13] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.1.post_attention_layernorm.weight", shape: (8192,), dtype: float32

  5%|████▋                                                                                              | 23/483 [00:09<02:10,  3.51it/s]
                                                                                                                                         
[2024-01-30 23:23:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.1.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

  5%|████▋                                                                                              | 23/483 [00:09<02:10,  3.51it/s]
                                                                                                                                         
[2024-01-30 23:23:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.1.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

  5%|████▋                                                                                              | 23/483 [00:09<02:10,  3.51it/s]
  5%|█████                                                                                              | 25/483 [00:09<01:38,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:23:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.1.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

  5%|█████                                                                                              | 25/483 [00:09<01:38,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:23:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.1.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

  5%|█████                                                                                              | 25/483 [00:09<01:38,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:23:13] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00002-of-00029.bin

  5%|█████                                                                                              | 25/483 [00:09<01:38,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:23:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.2.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

  5%|█████                                                                                              | 25/483 [00:11<01:38,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:23:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.2.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

  5%|█████                                                                                              | 25/483 [00:11<01:38,  4.65it/s]
  6%|█████▌                                                                                             | 27/483 [00:11<03:58,  1.91it/s]
                                                                                                                                         
[2024-01-30 23:23:16] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.2.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

  6%|█████▌                                                                                             | 27/483 [00:11<03:58,  1.91it/s]
                                                                                                                                         
[2024-01-30 23:23:16] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.2.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

  6%|█████▌                                                                                             | 27/483 [00:11<03:58,  1.91it/s]
  6%|█████▋                                                                                             | 28/483 [00:11<03:26,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:23:16] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.2.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

  6%|█████▋                                                                                             | 28/483 [00:11<03:26,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:23:16] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.2.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

  6%|█████▋                                                                                             | 28/483 [00:11<03:26,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:23:16] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00002-of-00029.bin

  6%|█████▋                                                                                             | 28/483 [00:11<03:26,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:23:16] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00001-of-00029.bin

  6%|█████▋                                                                                             | 28/483 [00:11<03:26,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:23:16] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00004-of-00029.bin

  6%|█████▋                                                                                             | 28/483 [00:11<03:26,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:23:17] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.10.input_layernorm.weight", shape: (8192,), dtype: float32

  6%|█████▋                                                                                             | 28/483 [00:13<03:26,  2.20it/s]
  6%|██████▏                                                                                            | 30/483 [00:13<04:36,  1.64it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.10.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

  6%|██████▏                                                                                            | 30/483 [00:13<04:36,  1.64it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.10.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

  6%|██████▏                                                                                            | 30/483 [00:13<04:36,  1.64it/s]
  6%|██████▎                                                                                            | 31/483 [00:13<03:58,  1.90it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.10.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

  6%|██████▎                                                                                            | 31/483 [00:14<03:58,  1.90it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.10.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

  6%|██████▎                                                                                            | 31/483 [00:14<03:58,  1.90it/s]
  7%|██████▌                                                                                            | 32/483 [00:14<03:59,  1.89it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.10.post_attention_layernorm.weight", shape: (8192,), dtype: float32

  7%|██████▌                                                                                            | 32/483 [00:14<03:59,  1.89it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.10.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

  7%|██████▌                                                                                            | 32/483 [00:14<03:59,  1.89it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.10.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

  7%|██████▌                                                                                            | 32/483 [00:14<03:59,  1.89it/s]
  7%|██████▉                                                                                            | 34/483 [00:14<02:38,  2.84it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.10.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

  7%|██████▉                                                                                            | 34/483 [00:14<02:38,  2.84it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.10.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

  7%|██████▉                                                                                            | 34/483 [00:14<02:38,  2.84it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.8.input_layernorm.weight", shape: (8192,), dtype: float32

  7%|██████▉                                                                                            | 34/483 [00:14<02:38,  2.84it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.8.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

  7%|██████▉                                                                                            | 34/483 [00:14<02:38,  2.84it/s]
                                                                                                                                         
[2024-01-30 23:23:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.8.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

  7%|██████▉                                                                                            | 34/483 [00:14<02:38,  2.84it/s]
  8%|███████▌                                                                                           | 37/483 [00:14<01:41,  4.38it/s]
                                                                                                                                         
[2024-01-30 23:23:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.8.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

  8%|███████▌                                                                                           | 37/483 [00:15<01:41,  4.38it/s]
                                                                                                                                         
[2024-01-30 23:23:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.8.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

  8%|███████▌                                                                                           | 37/483 [00:15<01:41,  4.38it/s]
  8%|███████▊                                                                                           | 38/483 [00:15<02:06,  3.52it/s]
                                                                                                                                         
[2024-01-30 23:23:19] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.8.post_attention_layernorm.weight", shape: (8192,), dtype: float32

  8%|███████▊                                                                                           | 38/483 [00:15<02:06,  3.52it/s]
                                                                                                                                         
[2024-01-30 23:23:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.8.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

  8%|███████▊                                                                                           | 38/483 [00:15<02:06,  3.52it/s]
                                                                                                                                         
[2024-01-30 23:23:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.8.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

  8%|███████▊                                                                                           | 38/483 [00:15<02:06,  3.52it/s]
                                                                                                                                         
[2024-01-30 23:23:19] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.9.input_layernorm.weight", shape: (8192,), dtype: float32

  8%|███████▊                                                                                           | 38/483 [00:15<02:06,  3.52it/s]
                                                                                                                                         
[2024-01-30 23:23:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.9.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

  8%|███████▊                                                                                           | 38/483 [00:15<02:06,  3.52it/s]
                                                                                                                                         
[2024-01-30 23:23:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.9.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

  8%|███████▊                                                                                           | 38/483 [00:15<02:06,  3.52it/s]
  9%|████████▌                                                                                          | 42/483 [00:15<01:17,  5.70it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.9.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

  9%|████████▌                                                                                          | 42/483 [00:15<01:17,  5.70it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.9.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

  9%|████████▌                                                                                          | 42/483 [00:15<01:17,  5.70it/s]
  9%|████████▊                                                                                          | 43/483 [00:15<01:42,  4.30it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.9.post_attention_layernorm.weight", shape: (8192,), dtype: float32

  9%|████████▊                                                                                          | 43/483 [00:15<01:42,  4.30it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.9.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

  9%|████████▊                                                                                          | 43/483 [00:16<01:42,  4.30it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.9.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

  9%|████████▊                                                                                          | 43/483 [00:16<01:42,  4.30it/s]
  9%|█████████▏                                                                                         | 45/483 [00:16<01:20,  5.43it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.9.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

  9%|█████████▏                                                                                         | 45/483 [00:16<01:20,  5.43it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.9.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

  9%|█████████▏                                                                                         | 45/483 [00:16<01:20,  5.43it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00004-of-00029.bin

  9%|█████████▏                                                                                         | 45/483 [00:16<01:20,  5.43it/s]
                                                                                                                                         
[2024-01-30 23:23:20] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00005-of-00029.bin

  9%|█████████▏                                                                                         | 45/483 [00:16<01:20,  5.43it/s]
                                                                                                                                         
[2024-01-30 23:23:22] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.11.input_layernorm.weight", shape: (8192,), dtype: float32

  9%|█████████▏                                                                                         | 45/483 [00:17<01:20,  5.43it/s]
 10%|█████████▋                                                                                         | 47/483 [00:17<02:42,  2.69it/s]
                                                                                                                                         
[2024-01-30 23:23:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.11.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 10%|█████████▋                                                                                         | 47/483 [00:17<02:42,  2.69it/s]
                                                                                                                                         
[2024-01-30 23:23:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.11.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 10%|█████████▋                                                                                         | 47/483 [00:17<02:42,  2.69it/s]
 10%|█████████▊                                                                                         | 48/483 [00:17<02:27,  2.96it/s]
                                                                                                                                         
[2024-01-30 23:23:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.11.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 10%|█████████▊                                                                                         | 48/483 [00:18<02:27,  2.96it/s]
                                                                                                                                         
[2024-01-30 23:23:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.11.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 10%|█████████▊                                                                                         | 48/483 [00:18<02:27,  2.96it/s]
 10%|██████████                                                                                         | 49/483 [00:18<02:44,  2.64it/s]
                                                                                                                                         
[2024-01-30 23:23:22] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.11.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 10%|██████████                                                                                         | 49/483 [00:18<02:44,  2.64it/s]
                                                                                                                                         
[2024-01-30 23:23:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.11.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 10%|██████████                                                                                         | 49/483 [00:18<02:44,  2.64it/s]
                                                                                                                                         
[2024-01-30 23:23:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.11.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 10%|██████████                                                                                         | 49/483 [00:18<02:44,  2.64it/s]
 11%|██████████▍                                                                                        | 51/483 [00:18<01:55,  3.74it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.11.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 11%|██████████▍                                                                                        | 51/483 [00:18<01:55,  3.74it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.11.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 11%|██████████▍                                                                                        | 51/483 [00:18<01:55,  3.74it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.12.input_layernorm.weight", shape: (8192,), dtype: float32

 11%|██████████▍                                                                                        | 51/483 [00:18<01:55,  3.74it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.12.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 11%|██████████▍                                                                                        | 51/483 [00:18<01:55,  3.74it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.12.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 11%|██████████▍                                                                                        | 51/483 [00:18<01:55,  3.74it/s]
 11%|███████████                                                                                        | 54/483 [00:18<01:20,  5.35it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.12.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 11%|███████████                                                                                        | 54/483 [00:19<01:20,  5.35it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.12.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 11%|███████████                                                                                        | 54/483 [00:19<01:20,  5.35it/s]
 11%|███████████▎                                                                                       | 55/483 [00:19<01:46,  4.02it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.12.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 11%|███████████▎                                                                                       | 55/483 [00:19<01:46,  4.02it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.12.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 11%|███████████▎                                                                                       | 55/483 [00:19<01:46,  4.02it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.12.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 11%|███████████▎                                                                                       | 55/483 [00:19<01:46,  4.02it/s]
 12%|███████████▋                                                                                       | 57/483 [00:19<01:20,  5.27it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.12.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 12%|███████████▋                                                                                       | 57/483 [00:19<01:20,  5.27it/s]
                                                                                                                                         
[2024-01-30 23:23:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.12.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 12%|███████████▋                                                                                       | 57/483 [00:19<01:20,  5.27it/s]
                                                                                                                                         
[2024-01-30 23:23:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.13.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 12%|███████████▋                                                                                       | 57/483 [00:20<01:20,  5.27it/s]
                                                                                                                                         
[2024-01-30 23:23:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.13.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 12%|███████████▋                                                                                       | 57/483 [00:20<01:20,  5.27it/s]
 12%|████████████                                                                                       | 59/483 [00:20<01:34,  4.49it/s]
                                                                                                                                         
[2024-01-30 23:23:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.13.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 12%|████████████                                                                                       | 59/483 [00:20<01:34,  4.49it/s]
                                                                                                                                         
[2024-01-30 23:23:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.13.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 12%|████████████                                                                                       | 59/483 [00:20<01:34,  4.49it/s]
 12%|████████████▎                                                                                      | 60/483 [00:20<01:27,  4.85it/s]
                                                                                                                                         
[2024-01-30 23:23:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.13.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 12%|████████████▎                                                                                      | 60/483 [00:20<01:27,  4.85it/s]
                                                                                                                                         
[2024-01-30 23:23:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.13.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 12%|████████████▎                                                                                      | 60/483 [00:20<01:27,  4.85it/s]
                                                                                                                                         
[2024-01-30 23:23:24] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00005-of-00029.bin

 12%|████████████▎                                                                                      | 60/483 [00:20<01:27,  4.85it/s]
                                                                                                                                         
[2024-01-30 23:23:24] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00006-of-00029.bin

 12%|████████████▎                                                                                      | 60/483 [00:20<01:27,  4.85it/s]
                                                                                                                                         
[2024-01-30 23:23:26] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.13.input_layernorm.weight", shape: (8192,), dtype: float32

 12%|████████████▎                                                                                      | 60/483 [00:21<01:27,  4.85it/s]
 13%|████████████▋                                                                                      | 62/483 [00:21<02:53,  2.42it/s]
                                                                                                                                         
[2024-01-30 23:23:26] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.13.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 13%|████████████▋                                                                                      | 62/483 [00:22<02:53,  2.42it/s]
                                                                                                                                         
[2024-01-30 23:23:26] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.13.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 13%|████████████▋                                                                                      | 62/483 [00:22<02:53,  2.42it/s]
 13%|████████████▉                                                                                      | 63/483 [00:22<02:34,  2.72it/s]
                                                                                                                                         
[2024-01-30 23:23:26] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.13.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 13%|████████████▉                                                                                      | 63/483 [00:22<02:34,  2.72it/s]
                                                                                                                                         
[2024-01-30 23:23:26] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.14.input_layernorm.weight", shape: (8192,), dtype: float32

 13%|████████████▉                                                                                      | 63/483 [00:22<02:34,  2.72it/s]
                                                                                                                                         
[2024-01-30 23:23:26] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.14.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 13%|████████████▉                                                                                      | 63/483 [00:22<02:34,  2.72it/s]
                                                                                                                                         
[2024-01-30 23:23:26] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.14.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 13%|████████████▉                                                                                      | 63/483 [00:22<02:34,  2.72it/s]
 14%|█████████████▌                                                                                     | 66/483 [00:22<01:36,  4.34it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.14.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 14%|█████████████▌                                                                                     | 66/483 [00:22<01:36,  4.34it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.14.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 14%|█████████████▌                                                                                     | 66/483 [00:22<01:36,  4.34it/s]
 14%|█████████████▋                                                                                     | 67/483 [00:22<01:59,  3.48it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.14.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 14%|█████████████▋                                                                                     | 67/483 [00:22<01:59,  3.48it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.14.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 14%|█████████████▋                                                                                     | 67/483 [00:22<01:59,  3.48it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.14.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 14%|█████████████▋                                                                                     | 67/483 [00:22<01:59,  3.48it/s]
 14%|██████████████▏                                                                                    | 69/483 [00:22<01:28,  4.67it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.14.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 14%|██████████████▏                                                                                    | 69/483 [00:22<01:28,  4.67it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.14.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 14%|██████████████▏                                                                                    | 69/483 [00:22<01:28,  4.67it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.15.input_layernorm.weight", shape: (8192,), dtype: float32

 14%|██████████████▏                                                                                    | 69/483 [00:22<01:28,  4.67it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.15.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 14%|██████████████▏                                                                                    | 69/483 [00:23<01:28,  4.67it/s]
                                                                                                                                         
[2024-01-30 23:23:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.15.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 14%|██████████████▏                                                                                    | 69/483 [00:23<01:28,  4.67it/s]
 15%|██████████████▊                                                                                    | 72/483 [00:23<01:05,  6.27it/s]
                                                                                                                                         
[2024-01-30 23:23:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.15.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 15%|██████████████▊                                                                                    | 72/483 [00:23<01:05,  6.27it/s]
                                                                                                                                         
[2024-01-30 23:23:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.15.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 15%|██████████████▊                                                                                    | 72/483 [00:23<01:05,  6.27it/s]
 15%|██████████████▉                                                                                    | 73/483 [00:23<01:33,  4.40it/s]
                                                                                                                                         
[2024-01-30 23:23:28] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.15.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 15%|██████████████▉                                                                                    | 73/483 [00:23<01:33,  4.40it/s]
                                                                                                                                         
[2024-01-30 23:23:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.15.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 15%|██████████████▉                                                                                    | 73/483 [00:23<01:33,  4.40it/s]
                                                                                                                                         
[2024-01-30 23:23:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.15.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 15%|██████████████▉                                                                                    | 73/483 [00:23<01:33,  4.40it/s]
 16%|███████████████▎                                                                                   | 75/483 [00:23<01:12,  5.65it/s]
                                                                                                                                         
[2024-01-30 23:23:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.15.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 16%|███████████████▎                                                                                   | 75/483 [00:23<01:12,  5.65it/s]
                                                                                                                                         
[2024-01-30 23:23:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.15.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 16%|███████████████▎                                                                                   | 75/483 [00:23<01:12,  5.65it/s]
                                                                                                                                         
[2024-01-30 23:23:28] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00007-of-00029.bin

 16%|███████████████▎                                                                                   | 75/483 [00:23<01:12,  5.65it/s]
                                                                                                                                         
[2024-01-30 23:23:30] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.16.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 16%|███████████████▎                                                                                   | 75/483 [00:26<01:12,  5.65it/s]
                                                                                                                                         
[2024-01-30 23:23:30] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.16.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 16%|███████████████▎                                                                                   | 75/483 [00:26<01:12,  5.65it/s]
 16%|███████████████▊                                                                                   | 77/483 [00:26<03:32,  1.91it/s]
                                                                                                                                         
[2024-01-30 23:23:30] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.16.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 16%|███████████████▊                                                                                   | 77/483 [00:26<03:32,  1.91it/s]
                                                                                                                                         
[2024-01-30 23:23:30] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.16.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 16%|███████████████▊                                                                                   | 77/483 [00:26<03:32,  1.91it/s]
 16%|███████████████▉                                                                                   | 78/483 [00:26<03:02,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.16.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 16%|███████████████▉                                                                                   | 78/483 [00:26<03:02,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.16.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 16%|███████████████▉                                                                                   | 78/483 [00:26<03:02,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.16.input_layernorm.weight", shape: (8192,), dtype: float32

 16%|███████████████▉                                                                                   | 78/483 [00:26<03:02,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.16.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 16%|███████████████▉                                                                                   | 78/483 [00:26<03:02,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.16.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 16%|███████████████▉                                                                                   | 78/483 [00:26<03:02,  2.22it/s]
 17%|████████████████▌                                                                                  | 81/483 [00:26<01:55,  3.47it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.16.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 17%|████████████████▌                                                                                  | 81/483 [00:26<01:55,  3.47it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.17.input_layernorm.weight", shape: (8192,), dtype: float32

 17%|████████████████▌                                                                                  | 81/483 [00:26<01:55,  3.47it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.17.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 17%|████████████████▌                                                                                  | 81/483 [00:26<01:55,  3.47it/s]
                                                                                                                                         
[2024-01-30 23:23:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.17.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 17%|████████████████▌                                                                                  | 81/483 [00:26<01:55,  3.47it/s]
 17%|█████████████████▏                                                                                 | 84/483 [00:27<01:20,  4.96it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.17.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 17%|█████████████████▏                                                                                 | 84/483 [00:27<01:20,  4.96it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.17.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 17%|█████████████████▏                                                                                 | 84/483 [00:27<01:20,  4.96it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.17.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 17%|█████████████████▏                                                                                 | 84/483 [00:27<01:20,  4.96it/s]
 18%|█████████████████▋                                                                                 | 86/483 [00:27<01:39,  4.01it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.17.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 18%|█████████████████▋                                                                                 | 86/483 [00:27<01:39,  4.01it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.17.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 18%|█████████████████▋                                                                                 | 86/483 [00:27<01:39,  4.01it/s]
 18%|█████████████████▊                                                                                 | 87/483 [00:27<01:29,  4.40it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.17.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 18%|█████████████████▊                                                                                 | 87/483 [00:27<01:29,  4.40it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.17.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 18%|█████████████████▊                                                                                 | 87/483 [00:27<01:29,  4.40it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.18.input_layernorm.weight", shape: (8192,), dtype: float32

 18%|█████████████████▊                                                                                 | 87/483 [00:27<01:29,  4.40it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.18.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 18%|█████████████████▊                                                                                 | 87/483 [00:28<01:29,  4.40it/s]
                                                                                                                                         
[2024-01-30 23:23:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.18.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 18%|█████████████████▊                                                                                 | 87/483 [00:28<01:29,  4.40it/s]
 19%|██████████████████▍                                                                                | 90/483 [00:28<01:10,  5.54it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.18.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 19%|██████████████████▍                                                                                | 90/483 [00:28<01:10,  5.54it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.18.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 19%|██████████████████▍                                                                                | 90/483 [00:28<01:10,  5.54it/s]
 19%|██████████████████▋                                                                                | 91/483 [00:28<01:48,  3.61it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.18.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 19%|██████████████████▋                                                                                | 91/483 [00:29<01:48,  3.61it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.18.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 19%|██████████████████▋                                                                                | 91/483 [00:29<01:48,  3.61it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.18.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 19%|██████████████████▋                                                                                | 91/483 [00:29<01:48,  3.61it/s]
 19%|███████████████████                                                                                | 93/483 [00:29<01:22,  4.74it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.18.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 19%|███████████████████                                                                                | 93/483 [00:29<01:22,  4.74it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.18.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 19%|███████████████████                                                                                | 93/483 [00:29<01:22,  4.74it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.19.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 19%|███████████████████                                                                                | 93/483 [00:29<01:22,  4.74it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.19.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 19%|███████████████████                                                                                | 93/483 [00:29<01:22,  4.74it/s]
 20%|███████████████████▍                                                                               | 95/483 [00:29<01:07,  5.72it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.19.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 20%|███████████████████▍                                                                               | 95/483 [00:29<01:07,  5.72it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.19.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 20%|███████████████████▍                                                                               | 95/483 [00:29<01:07,  5.72it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00006-of-00029.bin

 20%|███████████████████▍                                                                               | 95/483 [00:29<01:07,  5.72it/s]
                                                                                                                                         
[2024-01-30 23:23:33] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00007-of-00029.bin

 20%|███████████████████▍                                                                               | 95/483 [00:29<01:07,  5.72it/s]
                                                                                                                                         
[2024-01-30 23:23:34] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00008-of-00029.bin

 20%|███████████████████▍                                                                               | 95/483 [00:29<01:07,  5.72it/s]
                                                                                                                                         
[2024-01-30 23:23:35] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.19.input_layernorm.weight", shape: (8192,), dtype: float32

 20%|███████████████████▍                                                                               | 95/483 [00:31<01:07,  5.72it/s]
 20%|███████████████████▉                                                                               | 97/483 [00:31<02:33,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:23:35] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.19.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 20%|███████████████████▉                                                                               | 97/483 [00:31<02:33,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:23:35] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.19.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 20%|███████████████████▉                                                                               | 97/483 [00:31<02:33,  2.51it/s]
 20%|████████████████████                                                                               | 98/483 [00:31<02:18,  2.77it/s]
                                                                                                                                         
[2024-01-30 23:23:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.19.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 20%|████████████████████                                                                               | 98/483 [00:31<02:18,  2.77it/s]
                                                                                                                                         
[2024-01-30 23:23:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.19.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 20%|████████████████████                                                                               | 98/483 [00:31<02:18,  2.77it/s]
 20%|████████████████████▎                                                                              | 99/483 [00:31<02:32,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:23:36] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.19.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 20%|████████████████████▎                                                                              | 99/483 [00:31<02:32,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:23:36] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.20.input_layernorm.weight", shape: (8192,), dtype: float32

 20%|████████████████████▎                                                                              | 99/483 [00:31<02:32,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:23:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.20.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 20%|████████████████████▎                                                                              | 99/483 [00:32<02:32,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:23:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.20.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 20%|████████████████████▎                                                                              | 99/483 [00:32<02:32,  2.51it/s]
 21%|████████████████████▋                                                                             | 102/483 [00:32<01:33,  4.07it/s]
                                                                                                                                         
[2024-01-30 23:23:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.20.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 21%|████████████████████▋                                                                             | 102/483 [00:32<01:33,  4.07it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.20.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 21%|████████████████████▋                                                                             | 102/483 [00:32<01:33,  4.07it/s]
 21%|████████████████████▉                                                                             | 103/483 [00:32<01:53,  3.33it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.20.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 21%|████████████████████▉                                                                             | 103/483 [00:32<01:53,  3.33it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.20.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 21%|████████████████████▉                                                                             | 103/483 [00:32<01:53,  3.33it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.20.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 21%|████████████████████▉                                                                             | 103/483 [00:32<01:53,  3.33it/s]
 22%|█████████████████████▎                                                                            | 105/483 [00:32<01:23,  4.51it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.20.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 22%|█████████████████████▎                                                                            | 105/483 [00:32<01:23,  4.51it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.20.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 22%|█████████████████████▎                                                                            | 105/483 [00:32<01:23,  4.51it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.21.input_layernorm.weight", shape: (8192,), dtype: float32

 22%|█████████████████████▎                                                                            | 105/483 [00:32<01:23,  4.51it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.21.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 22%|█████████████████████▎                                                                            | 105/483 [00:32<01:23,  4.51it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.21.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 22%|█████████████████████▎                                                                            | 105/483 [00:32<01:23,  4.51it/s]
 22%|█████████████████████▉                                                                            | 108/483 [00:32<01:00,  6.19it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.21.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 22%|█████████████████████▉                                                                            | 108/483 [00:33<01:00,  6.19it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.21.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 22%|█████████████████████▉                                                                            | 108/483 [00:33<01:00,  6.19it/s]
 23%|██████████████████████                                                                            | 109/483 [00:33<01:23,  4.45it/s]
                                                                                                                                         
[2024-01-30 23:23:37] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.21.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 23%|██████████████████████                                                                            | 109/483 [00:33<01:23,  4.45it/s]
                                                                                                                                         
[2024-01-30 23:23:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.21.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 23%|██████████████████████                                                                            | 109/483 [00:33<01:23,  4.45it/s]
                                                                                                                                         
[2024-01-30 23:23:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.21.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 23%|██████████████████████                                                                            | 109/483 [00:33<01:23,  4.45it/s]
 23%|██████████████████████▌                                                                           | 111/483 [00:33<01:05,  5.71it/s]
                                                                                                                                         
[2024-01-30 23:23:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.21.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 23%|██████████████████████▌                                                                           | 111/483 [00:33<01:05,  5.71it/s]
                                                                                                                                         
[2024-01-30 23:23:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.21.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 23%|██████████████████████▌                                                                           | 111/483 [00:33<01:05,  5.71it/s]
                                                                                                                                         
[2024-01-30 23:23:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.22.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 23%|██████████████████████▌                                                                           | 111/483 [00:33<01:05,  5.71it/s]
                                                                                                                                         
[2024-01-30 23:23:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.22.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 23%|██████████████████████▌                                                                           | 111/483 [00:33<01:05,  5.71it/s]
 23%|██████████████████████▉                                                                           | 113/483 [00:33<00:53,  6.86it/s]
                                                                                                                                         
[2024-01-30 23:23:38] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00008-of-00029.bin

 23%|██████████████████████▉                                                                           | 113/483 [00:33<00:53,  6.86it/s]
                                                                                                                                         
[2024-01-30 23:23:38] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00002-of-00029.bin

 23%|██████████████████████▉                                                                           | 113/483 [00:33<00:53,  6.86it/s]
                                                                                                                                         
[2024-01-30 23:23:39] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.2.input_layernorm.weight", shape: (8192,), dtype: float32

 23%|██████████████████████▉                                                                           | 113/483 [00:35<00:53,  6.86it/s]
                                                                                                                                         
[2024-01-30 23:23:39] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.2.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 23%|██████████████████████▉                                                                           | 113/483 [00:35<00:53,  6.86it/s]
                                                                                                                                         
[2024-01-30 23:23:39] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.2.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 23%|██████████████████████▉                                                                           | 113/483 [00:35<00:53,  6.86it/s]
 24%|███████████████████████▎                                                                          | 115/483 [00:35<02:11,  2.81it/s]
                                                                                                                                         
[2024-01-30 23:23:39] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.2.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 24%|███████████████████████▎                                                                          | 115/483 [00:35<02:11,  2.81it/s]
                                                                                                                                         
[2024-01-30 23:23:39] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.3.input_layernorm.weight", shape: (8192,), dtype: float32

 24%|███████████████████████▎                                                                          | 115/483 [00:35<02:11,  2.81it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.3.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 24%|███████████████████████▎                                                                          | 115/483 [00:35<02:11,  2.81it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.3.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 24%|███████████████████████▎                                                                          | 115/483 [00:35<02:11,  2.81it/s]
 24%|███████████████████████▉                                                                          | 118/483 [00:35<01:27,  4.17it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.3.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 24%|███████████████████████▉                                                                          | 118/483 [00:36<01:27,  4.17it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.3.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 24%|███████████████████████▉                                                                          | 118/483 [00:36<01:27,  4.17it/s]
 25%|████████████████████████▏                                                                         | 119/483 [00:36<01:45,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.3.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 25%|████████████████████████▏                                                                         | 119/483 [00:36<01:45,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.3.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 25%|████████████████████████▏                                                                         | 119/483 [00:36<01:45,  3.46it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.3.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 25%|████████████████████████▏                                                                         | 119/483 [00:36<01:45,  3.46it/s]
 25%|████████████████████████▌                                                                         | 121/483 [00:36<01:19,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.3.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 25%|████████████████████████▌                                                                         | 121/483 [00:36<01:19,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.3.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 25%|████████████████████████▌                                                                         | 121/483 [00:36<01:19,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.4.input_layernorm.weight", shape: (8192,), dtype: float32

 25%|████████████████████████▌                                                                         | 121/483 [00:36<01:19,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.4.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 25%|████████████████████████▌                                                                         | 121/483 [00:36<01:19,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:23:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.4.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 25%|████████████████████████▌                                                                         | 121/483 [00:36<01:19,  4.56it/s]
 26%|█████████████████████████▏                                                                        | 124/483 [00:36<00:58,  6.15it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.4.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 26%|█████████████████████████▏                                                                        | 124/483 [00:37<00:58,  6.15it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.4.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 26%|█████████████████████████▏                                                                        | 124/483 [00:37<00:58,  6.15it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.4.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 26%|█████████████████████████▏                                                                        | 124/483 [00:37<00:58,  6.15it/s]
 26%|█████████████████████████▌                                                                        | 126/483 [00:37<01:10,  5.08it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.4.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 26%|█████████████████████████▌                                                                        | 126/483 [00:37<01:10,  5.08it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.4.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 26%|█████████████████████████▌                                                                        | 126/483 [00:37<01:10,  5.08it/s]
 26%|█████████████████████████▊                                                                        | 127/483 [00:37<01:04,  5.49it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.4.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 26%|█████████████████████████▊                                                                        | 127/483 [00:37<01:04,  5.49it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.4.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 26%|█████████████████████████▊                                                                        | 127/483 [00:37<01:04,  5.49it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.5.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 26%|█████████████████████████▊                                                                        | 127/483 [00:37<01:04,  5.49it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.5.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 26%|█████████████████████████▊                                                                        | 127/483 [00:37<01:04,  5.49it/s]
 27%|██████████████████████████▏                                                                       | 129/483 [00:37<00:53,  6.65it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.5.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 27%|██████████████████████████▏                                                                       | 129/483 [00:37<00:53,  6.65it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.5.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 27%|██████████████████████████▏                                                                       | 129/483 [00:37<00:53,  6.65it/s]
                                                                                                                                         
[2024-01-30 23:23:41] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00002-of-00029.bin

 27%|██████████████████████████▏                                                                       | 129/483 [00:37<00:53,  6.65it/s]
                                                                                                                                         
[2024-01-30 23:23:42] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00009-of-00029.bin

 27%|██████████████████████████▏                                                                       | 129/483 [00:37<00:53,  6.65it/s]
                                                                                                                                         
[2024-01-30 23:23:43] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.22.input_layernorm.weight", shape: (8192,), dtype: float32

 27%|██████████████████████████▏                                                                       | 129/483 [00:39<00:53,  6.65it/s]
 27%|██████████████████████████▌                                                                       | 131/483 [00:39<02:10,  2.70it/s]
                                                                                                                                         
[2024-01-30 23:23:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.22.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 27%|██████████████████████████▌                                                                       | 131/483 [00:39<02:10,  2.70it/s]
                                                                                                                                         
[2024-01-30 23:23:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.22.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 27%|██████████████████████████▌                                                                       | 131/483 [00:39<02:10,  2.70it/s]
 27%|██████████████████████████▊                                                                       | 132/483 [00:39<01:59,  2.95it/s]
                                                                                                                                         
[2024-01-30 23:23:44] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.22.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 27%|██████████████████████████▊                                                                       | 132/483 [00:39<01:59,  2.95it/s]
                                                                                                                                         
[2024-01-30 23:23:44] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.22.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 27%|██████████████████████████▊                                                                       | 132/483 [00:39<01:59,  2.95it/s]
 28%|██████████████████████████▉                                                                       | 133/483 [00:39<02:13,  2.62it/s]
                                                                                                                                         
[2024-01-30 23:23:44] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.22.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 28%|██████████████████████████▉                                                                       | 133/483 [00:39<02:13,  2.62it/s]
                                                                                                                                         
[2024-01-30 23:23:44] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.22.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 28%|██████████████████████████▉                                                                       | 133/483 [00:39<02:13,  2.62it/s]
                                                                                                                                         
[2024-01-30 23:23:44] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.22.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 28%|██████████████████████████▉                                                                       | 133/483 [00:39<02:13,  2.62it/s]
                                                                                                                                         
[2024-01-30 23:23:44] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.23.input_layernorm.weight", shape: (8192,), dtype: float32

 28%|██████████████████████████▉                                                                       | 133/483 [00:39<02:13,  2.62it/s]
 28%|███████████████████████████▌                                                                      | 136/483 [00:39<01:15,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:23:44] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.23.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 28%|███████████████████████████▌                                                                      | 136/483 [00:40<01:15,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:23:44] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.23.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 28%|███████████████████████████▌                                                                      | 136/483 [00:40<01:15,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.23.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 28%|███████████████████████████▌                                                                      | 136/483 [00:40<01:15,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.23.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 28%|███████████████████████████▌                                                                      | 136/483 [00:40<01:15,  4.58it/s]
 29%|████████████████████████████                                                                      | 138/483 [00:40<01:30,  3.83it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.23.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 29%|████████████████████████████                                                                      | 138/483 [00:40<01:30,  3.83it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.23.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 29%|████████████████████████████                                                                      | 138/483 [00:40<01:30,  3.83it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.23.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 29%|████████████████████████████                                                                      | 138/483 [00:40<01:30,  3.83it/s]
 29%|████████████████████████████▍                                                                     | 140/483 [00:40<01:09,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.23.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 29%|████████████████████████████▍                                                                     | 140/483 [00:40<01:09,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.23.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 29%|████████████████████████████▍                                                                     | 140/483 [00:40<01:09,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.24.input_layernorm.weight", shape: (8192,), dtype: float32

 29%|████████████████████████████▍                                                                     | 140/483 [00:40<01:09,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.24.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 29%|████████████████████████████▍                                                                     | 140/483 [00:41<01:09,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:23:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.24.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 29%|████████████████████████████▍                                                                     | 140/483 [00:41<01:09,  4.95it/s]
 30%|█████████████████████████████                                                                     | 143/483 [00:41<00:52,  6.50it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.24.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 30%|█████████████████████████████                                                                     | 143/483 [00:41<00:52,  6.50it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.24.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 30%|█████████████████████████████                                                                     | 143/483 [00:41<00:52,  6.50it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.24.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 30%|█████████████████████████████                                                                     | 143/483 [00:41<00:52,  6.50it/s]
 30%|█████████████████████████████▍                                                                    | 145/483 [00:41<01:04,  5.22it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.24.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 30%|█████████████████████████████▍                                                                    | 145/483 [00:41<01:04,  5.22it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.24.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 30%|█████████████████████████████▍                                                                    | 145/483 [00:41<01:04,  5.22it/s]
 30%|█████████████████████████████▌                                                                    | 146/483 [00:41<00:59,  5.62it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.24.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 30%|█████████████████████████████▌                                                                    | 146/483 [00:41<00:59,  5.62it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.24.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 30%|█████████████████████████████▌                                                                    | 146/483 [00:41<00:59,  5.62it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00009-of-00029.bin

 30%|█████████████████████████████▌                                                                    | 146/483 [00:41<00:59,  5.62it/s]
                                                                                                                                         
[2024-01-30 23:23:46] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00010-of-00029.bin

 30%|█████████████████████████████▌                                                                    | 146/483 [00:41<00:59,  5.62it/s]
                                                                                                                                         
[2024-01-30 23:23:47] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.25.input_layernorm.weight", shape: (8192,), dtype: float32

 30%|█████████████████████████████▌                                                                    | 146/483 [00:43<00:59,  5.62it/s]
 31%|██████████████████████████████                                                                    | 148/483 [00:43<02:07,  2.63it/s]
                                                                                                                                         
[2024-01-30 23:23:47] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.25.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 31%|██████████████████████████████                                                                    | 148/483 [00:43<02:07,  2.63it/s]
                                                                                                                                         
[2024-01-30 23:23:47] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.25.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 31%|██████████████████████████████                                                                    | 148/483 [00:43<02:07,  2.63it/s]
 31%|██████████████████████████████▏                                                                   | 149/483 [00:43<01:55,  2.89it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.25.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 31%|██████████████████████████████▏                                                                   | 149/483 [00:44<01:55,  2.89it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.25.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 31%|██████████████████████████████▏                                                                   | 149/483 [00:44<01:55,  2.89it/s]
 31%|██████████████████████████████▍                                                                   | 150/483 [00:44<02:09,  2.57it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.25.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 31%|██████████████████████████████▍                                                                   | 150/483 [00:44<02:09,  2.57it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.25.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 31%|██████████████████████████████▍                                                                   | 150/483 [00:44<02:09,  2.57it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.25.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 31%|██████████████████████████████▍                                                                   | 150/483 [00:44<02:09,  2.57it/s]
 31%|██████████████████████████████▊                                                                   | 152/483 [00:44<01:29,  3.70it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.25.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 31%|██████████████████████████████▊                                                                   | 152/483 [00:44<01:29,  3.70it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.25.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 31%|██████████████████████████████▊                                                                   | 152/483 [00:44<01:29,  3.70it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.26.input_layernorm.weight", shape: (8192,), dtype: float32

 31%|██████████████████████████████▊                                                                   | 152/483 [00:44<01:29,  3.70it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.26.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 31%|██████████████████████████████▊                                                                   | 152/483 [00:44<01:29,  3.70it/s]
                                                                                                                                         
[2024-01-30 23:23:48] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.26.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 31%|██████████████████████████████▊                                                                   | 152/483 [00:44<01:29,  3.70it/s]
 32%|███████████████████████████████▍                                                                  | 155/483 [00:44<01:01,  5.36it/s]
                                                                                                                                         
[2024-01-30 23:23:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.26.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 32%|███████████████████████████████▍                                                                  | 155/483 [00:45<01:01,  5.36it/s]
                                                                                                                                         
[2024-01-30 23:23:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.26.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 32%|███████████████████████████████▍                                                                  | 155/483 [00:45<01:01,  5.36it/s]
 32%|███████████████████████████████▋                                                                  | 156/483 [00:45<01:24,  3.89it/s]
                                                                                                                                         
[2024-01-30 23:23:49] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.26.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 32%|███████████████████████████████▋                                                                  | 156/483 [00:45<01:24,  3.89it/s]
                                                                                                                                         
[2024-01-30 23:23:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.26.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 32%|███████████████████████████████▋                                                                  | 156/483 [00:45<01:24,  3.89it/s]
                                                                                                                                         
[2024-01-30 23:23:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.26.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 32%|███████████████████████████████▋                                                                  | 156/483 [00:45<01:24,  3.89it/s]
 33%|████████████████████████████████                                                                  | 158/483 [00:45<01:02,  5.18it/s]
                                                                                                                                         
[2024-01-30 23:23:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.26.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 33%|████████████████████████████████                                                                  | 158/483 [00:45<01:02,  5.18it/s]
                                                                                                                                         
[2024-01-30 23:23:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.26.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 33%|████████████████████████████████                                                                  | 158/483 [00:45<01:02,  5.18it/s]
                                                                                                                                         
[2024-01-30 23:23:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.27.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 33%|████████████████████████████████                                                                  | 158/483 [00:45<01:02,  5.18it/s]
                                                                                                                                         
[2024-01-30 23:23:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.27.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 33%|████████████████████████████████                                                                  | 158/483 [00:45<01:02,  5.18it/s]
 33%|████████████████████████████████▍                                                                 | 160/483 [00:45<01:13,  4.37it/s]
                                                                                                                                         
[2024-01-30 23:23:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.27.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 33%|████████████████████████████████▍                                                                 | 160/483 [00:45<01:13,  4.37it/s]
                                                                                                                                         
[2024-01-30 23:23:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.27.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 33%|████████████████████████████████▍                                                                 | 160/483 [00:45<01:13,  4.37it/s]
 33%|████████████████████████████████▋                                                                 | 161/483 [00:45<01:07,  4.78it/s]
                                                                                                                                         
[2024-01-30 23:23:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.27.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 33%|████████████████████████████████▋                                                                 | 161/483 [00:46<01:07,  4.78it/s]
                                                                                                                                         
[2024-01-30 23:23:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.27.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 33%|████████████████████████████████▋                                                                 | 161/483 [00:46<01:07,  4.78it/s]
                                                                                                                                         
[2024-01-30 23:23:50] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00010-of-00029.bin

 33%|████████████████████████████████▋                                                                 | 161/483 [00:46<01:07,  4.78it/s]
                                                                                                                                         
[2024-01-30 23:23:50] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00011-of-00029.bin

 33%|████████████████████████████████▋                                                                 | 161/483 [00:46<01:07,  4.78it/s]
                                                                                                                                         
[2024-01-30 23:23:52] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.27.input_layernorm.weight", shape: (8192,), dtype: float32

 33%|████████████████████████████████▋                                                                 | 161/483 [00:47<01:07,  4.78it/s]
 34%|█████████████████████████████████                                                                 | 163/483 [00:47<02:17,  2.33it/s]
                                                                                                                                         
[2024-01-30 23:23:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.27.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 34%|█████████████████████████████████                                                                 | 163/483 [00:47<02:17,  2.33it/s]
                                                                                                                                         
[2024-01-30 23:23:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.27.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 34%|█████████████████████████████████                                                                 | 163/483 [00:47<02:17,  2.33it/s]
 34%|█████████████████████████████████▎                                                                | 164/483 [00:47<02:01,  2.62it/s]
                                                                                                                                         
[2024-01-30 23:23:52] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.27.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 34%|█████████████████████████████████▎                                                                | 164/483 [00:47<02:01,  2.62it/s]
                                                                                                                                         
[2024-01-30 23:23:52] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.28.input_layernorm.weight", shape: (8192,), dtype: float32

 34%|█████████████████████████████████▎                                                                | 164/483 [00:47<02:01,  2.62it/s]
                                                                                                                                         
[2024-01-30 23:23:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.28.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 34%|█████████████████████████████████▎                                                                | 164/483 [00:48<02:01,  2.62it/s]
                                                                                                                                         
[2024-01-30 23:23:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.28.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 34%|█████████████████████████████████▎                                                                | 164/483 [00:48<02:01,  2.62it/s]
 35%|█████████████████████████████████▉                                                                | 167/483 [00:48<01:14,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.28.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 35%|█████████████████████████████████▉                                                                | 167/483 [00:48<01:14,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.28.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 35%|█████████████████████████████████▉                                                                | 167/483 [00:48<01:14,  4.21it/s]
 35%|██████████████████████████████████                                                                | 168/483 [00:48<01:41,  3.10it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.28.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 35%|██████████████████████████████████                                                                | 168/483 [00:48<01:41,  3.10it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.28.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 35%|██████████████████████████████████                                                                | 168/483 [00:48<01:41,  3.10it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.28.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 35%|██████████████████████████████████                                                                | 168/483 [00:48<01:41,  3.10it/s]
 35%|██████████████████████████████████▍                                                               | 170/483 [00:48<01:14,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.28.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 35%|██████████████████████████████████▍                                                               | 170/483 [00:48<01:14,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.28.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 35%|██████████████████████████████████▍                                                               | 170/483 [00:48<01:14,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.29.input_layernorm.weight", shape: (8192,), dtype: float32

 35%|██████████████████████████████████▍                                                               | 170/483 [00:49<01:14,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.29.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 35%|██████████████████████████████████▍                                                               | 170/483 [00:49<01:14,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:23:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.29.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 35%|██████████████████████████████████▍                                                               | 170/483 [00:49<01:14,  4.18it/s]
 36%|███████████████████████████████████                                                               | 173/483 [00:49<00:53,  5.76it/s]
                                                                                                                                         
[2024-01-30 23:23:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.29.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 36%|███████████████████████████████████                                                               | 173/483 [00:49<00:53,  5.76it/s]
                                                                                                                                         
[2024-01-30 23:23:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.29.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 36%|███████████████████████████████████                                                               | 173/483 [00:49<00:53,  5.76it/s]
 36%|███████████████████████████████████▎                                                              | 174/483 [00:49<01:21,  3.81it/s]
                                                                                                                                         
[2024-01-30 23:23:54] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.29.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 36%|███████████████████████████████████▎                                                              | 174/483 [00:49<01:21,  3.81it/s]
                                                                                                                                         
[2024-01-30 23:23:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.29.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 36%|███████████████████████████████████▎                                                              | 174/483 [00:50<01:21,  3.81it/s]
                                                                                                                                         
[2024-01-30 23:23:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.29.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 36%|███████████████████████████████████▎                                                              | 174/483 [00:50<01:21,  3.81it/s]
 36%|███████████████████████████████████▋                                                              | 176/483 [00:50<01:01,  5.00it/s]
                                                                                                                                         
[2024-01-30 23:23:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.29.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 36%|███████████████████████████████████▋                                                              | 176/483 [00:50<01:01,  5.00it/s]
                                                                                                                                         
[2024-01-30 23:23:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.29.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 36%|███████████████████████████████████▋                                                              | 176/483 [00:50<01:01,  5.00it/s]
                                                                                                                                         
[2024-01-30 23:23:54] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00012-of-00029.bin

 36%|███████████████████████████████████▋                                                              | 176/483 [00:50<01:01,  5.00it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.30.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 36%|███████████████████████████████████▋                                                              | 176/483 [00:52<01:01,  5.00it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.30.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 36%|███████████████████████████████████▋                                                              | 176/483 [00:52<01:01,  5.00it/s]
 37%|████████████████████████████████████                                                              | 178/483 [00:52<02:55,  1.74it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.30.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 37%|████████████████████████████████████                                                              | 178/483 [00:52<02:55,  1.74it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.30.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 37%|████████████████████████████████████                                                              | 178/483 [00:52<02:55,  1.74it/s]
 37%|████████████████████████████████████▎                                                             | 179/483 [00:52<02:29,  2.03it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.30.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 37%|████████████████████████████████████▎                                                             | 179/483 [00:52<02:29,  2.03it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.30.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 37%|████████████████████████████████████▎                                                             | 179/483 [00:52<02:29,  2.03it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.30.input_layernorm.weight", shape: (8192,), dtype: float32

 37%|████████████████████████████████████▎                                                             | 179/483 [00:52<02:29,  2.03it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.30.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 37%|████████████████████████████████████▎                                                             | 179/483 [00:53<02:29,  2.03it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.30.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 37%|████████████████████████████████████▎                                                             | 179/483 [00:53<02:29,  2.03it/s]
 38%|████████████████████████████████████▉                                                             | 182/483 [00:53<01:38,  3.06it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.30.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 38%|████████████████████████████████████▉                                                             | 182/483 [00:53<01:38,  3.06it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.31.input_layernorm.weight", shape: (8192,), dtype: float32

 38%|████████████████████████████████████▉                                                             | 182/483 [00:53<01:38,  3.06it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.31.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 38%|████████████████████████████████████▉                                                             | 182/483 [00:53<01:38,  3.06it/s]
                                                                                                                                         
[2024-01-30 23:23:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.31.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 38%|████████████████████████████████████▉                                                             | 182/483 [00:53<01:38,  3.06it/s]
 38%|█████████████████████████████████████▌                                                            | 185/483 [00:53<01:11,  4.19it/s]
                                                                                                                                         
[2024-01-30 23:23:58] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.31.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 38%|█████████████████████████████████████▌                                                            | 185/483 [00:54<01:11,  4.19it/s]
                                                                                                                                         
[2024-01-30 23:23:58] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.31.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 38%|█████████████████████████████████████▌                                                            | 185/483 [00:54<01:11,  4.19it/s]
 39%|█████████████████████████████████████▋                                                            | 186/483 [00:54<01:41,  2.93it/s]
                                                                                                                                         
[2024-01-30 23:23:58] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.31.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 39%|█████████████████████████████████████▋                                                            | 186/483 [00:54<01:41,  2.93it/s]
                                                                                                                                         
[2024-01-30 23:23:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.31.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 39%|█████████████████████████████████████▋                                                            | 186/483 [00:54<01:41,  2.93it/s]
                                                                                                                                         
[2024-01-30 23:23:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.31.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 39%|█████████████████████████████████████▋                                                            | 186/483 [00:54<01:41,  2.93it/s]
 39%|██████████████████████████████████████▏                                                           | 188/483 [00:54<01:17,  3.82it/s]
                                                                                                                                         
[2024-01-30 23:23:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.31.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 39%|██████████████████████████████████████▏                                                           | 188/483 [00:54<01:17,  3.82it/s]
                                                                                                                                         
[2024-01-30 23:23:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.31.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 39%|██████████████████████████████████████▏                                                           | 188/483 [00:54<01:17,  3.82it/s]
                                                                                                                                         
[2024-01-30 23:23:59] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.32.input_layernorm.weight", shape: (8192,), dtype: float32

 39%|██████████████████████████████████████▏                                                           | 188/483 [00:54<01:17,  3.82it/s]
                                                                                                                                         
[2024-01-30 23:23:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.32.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 39%|██████████████████████████████████████▏                                                           | 188/483 [00:54<01:17,  3.82it/s]
                                                                                                                                         
[2024-01-30 23:23:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.32.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 39%|██████████████████████████████████████▏                                                           | 188/483 [00:54<01:17,  3.82it/s]
 40%|██████████████████████████████████████▊                                                           | 191/483 [00:54<00:59,  4.91it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.32.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 40%|██████████████████████████████████████▊                                                           | 191/483 [00:55<00:59,  4.91it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.32.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 40%|██████████████████████████████████████▊                                                           | 191/483 [00:55<00:59,  4.91it/s]
 40%|██████████████████████████████████████▉                                                           | 192/483 [00:55<01:29,  3.23it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.32.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 40%|██████████████████████████████████████▉                                                           | 192/483 [00:55<01:29,  3.23it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.32.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 40%|██████████████████████████████████████▉                                                           | 192/483 [00:56<01:29,  3.23it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.32.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 40%|██████████████████████████████████████▉                                                           | 192/483 [00:56<01:29,  3.23it/s]
 40%|███████████████████████████████████████▎                                                          | 194/483 [00:56<01:08,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.32.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 40%|███████████████████████████████████████▎                                                          | 194/483 [00:56<01:08,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.32.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 40%|███████████████████████████████████████▎                                                          | 194/483 [00:56<01:08,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.33.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 40%|███████████████████████████████████████▎                                                          | 194/483 [00:56<01:08,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.33.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 40%|███████████████████████████████████████▎                                                          | 194/483 [00:56<01:08,  4.21it/s]
 41%|███████████████████████████████████████▊                                                          | 196/483 [00:56<00:56,  5.09it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.33.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 41%|███████████████████████████████████████▊                                                          | 196/483 [00:56<00:56,  5.09it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.33.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 41%|███████████████████████████████████████▊                                                          | 196/483 [00:56<00:56,  5.09it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00011-of-00029.bin

 41%|███████████████████████████████████████▊                                                          | 196/483 [00:56<00:56,  5.09it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00012-of-00029.bin

 41%|███████████████████████████████████████▊                                                          | 196/483 [00:56<00:56,  5.09it/s]
                                                                                                                                         
[2024-01-30 23:24:00] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00013-of-00029.bin

 41%|███████████████████████████████████████▊                                                          | 196/483 [00:56<00:56,  5.09it/s]
                                                                                                                                         
[2024-01-30 23:24:02] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.33.input_layernorm.weight", shape: (8192,), dtype: float32

 41%|███████████████████████████████████████▊                                                          | 196/483 [00:58<00:56,  5.09it/s]
 41%|████████████████████████████████████████▏                                                         | 198/483 [00:58<02:01,  2.35it/s]
                                                                                                                                         
[2024-01-30 23:24:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.33.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 41%|████████████████████████████████████████▏                                                         | 198/483 [00:58<02:01,  2.35it/s]
                                                                                                                                         
[2024-01-30 23:24:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.33.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 41%|████████████████████████████████████████▏                                                         | 198/483 [00:58<02:01,  2.35it/s]
 41%|████████████████████████████████████████▍                                                         | 199/483 [00:58<01:50,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.33.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 41%|████████████████████████████████████████▍                                                         | 199/483 [00:58<01:50,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.33.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 41%|████████████████████████████████████████▍                                                         | 199/483 [00:58<01:50,  2.56it/s]
 41%|████████████████████████████████████████▌                                                         | 200/483 [00:58<02:05,  2.26it/s]
                                                                                                                                         
[2024-01-30 23:24:03] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.33.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 41%|████████████████████████████████████████▌                                                         | 200/483 [00:59<02:05,  2.26it/s]
                                                                                                                                         
[2024-01-30 23:24:03] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.34.input_layernorm.weight", shape: (8192,), dtype: float32

 41%|████████████████████████████████████████▌                                                         | 200/483 [00:59<02:05,  2.26it/s]
                                                                                                                                         
[2024-01-30 23:24:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.34.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 41%|████████████████████████████████████████▌                                                         | 200/483 [00:59<02:05,  2.26it/s]
                                                                                                                                         
[2024-01-30 23:24:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.34.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 41%|████████████████████████████████████████▌                                                         | 200/483 [00:59<02:05,  2.26it/s]
 42%|█████████████████████████████████████████▏                                                        | 203/483 [00:59<01:16,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.34.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 42%|█████████████████████████████████████████▏                                                        | 203/483 [00:59<01:16,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.34.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 42%|█████████████████████████████████████████▏                                                        | 203/483 [00:59<01:16,  3.66it/s]
 42%|█████████████████████████████████████████▍                                                        | 204/483 [00:59<01:37,  2.85it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.34.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 42%|█████████████████████████████████████████▍                                                        | 204/483 [00:59<01:37,  2.85it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.34.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 42%|█████████████████████████████████████████▍                                                        | 204/483 [01:00<01:37,  2.85it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.34.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 42%|█████████████████████████████████████████▍                                                        | 204/483 [01:00<01:37,  2.85it/s]
 43%|█████████████████████████████████████████▊                                                        | 206/483 [01:00<01:10,  3.93it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.34.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 43%|█████████████████████████████████████████▊                                                        | 206/483 [01:00<01:10,  3.93it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.34.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 43%|█████████████████████████████████████████▊                                                        | 206/483 [01:00<01:10,  3.93it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.35.input_layernorm.weight", shape: (8192,), dtype: float32

 43%|█████████████████████████████████████████▊                                                        | 206/483 [01:00<01:10,  3.93it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.35.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 43%|█████████████████████████████████████████▊                                                        | 206/483 [01:00<01:10,  3.93it/s]
                                                                                                                                         
[2024-01-30 23:24:04] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.35.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 43%|█████████████████████████████████████████▊                                                        | 206/483 [01:00<01:10,  3.93it/s]
 43%|██████████████████████████████████████████▍                                                       | 209/483 [01:00<00:52,  5.18it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.35.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 43%|██████████████████████████████████████████▍                                                       | 209/483 [01:01<00:52,  5.18it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.35.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 43%|██████████████████████████████████████████▍                                                       | 209/483 [01:01<00:52,  5.18it/s]
 43%|██████████████████████████████████████████▌                                                       | 210/483 [01:01<01:16,  3.56it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.35.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 43%|██████████████████████████████████████████▌                                                       | 210/483 [01:01<01:16,  3.56it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.35.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 43%|██████████████████████████████████████████▌                                                       | 210/483 [01:01<01:16,  3.56it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.35.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 43%|██████████████████████████████████████████▌                                                       | 210/483 [01:01<01:16,  3.56it/s]
 44%|███████████████████████████████████████████                                                       | 212/483 [01:01<00:57,  4.71it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.35.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 44%|███████████████████████████████████████████                                                       | 212/483 [01:01<00:57,  4.71it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.35.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 44%|███████████████████████████████████████████                                                       | 212/483 [01:01<00:57,  4.71it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.36.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 44%|███████████████████████████████████████████                                                       | 212/483 [01:01<00:57,  4.71it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.36.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 44%|███████████████████████████████████████████                                                       | 212/483 [01:01<00:57,  4.71it/s]
 44%|███████████████████████████████████████████▍                                                      | 214/483 [01:01<00:45,  5.89it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00013-of-00029.bin

 44%|███████████████████████████████████████████▍                                                      | 214/483 [01:01<00:45,  5.89it/s]
                                                                                                                                         
[2024-01-30 23:24:05] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00014-of-00029.bin

 44%|███████████████████████████████████████████▍                                                      | 214/483 [01:01<00:45,  5.89it/s]
                                                                                                                                         
[2024-01-30 23:24:07] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.36.input_layernorm.weight", shape: (8192,), dtype: float32

 44%|███████████████████████████████████████████▍                                                      | 214/483 [01:03<00:45,  5.89it/s]
 45%|███████████████████████████████████████████▌                                                      | 215/483 [01:03<01:59,  2.25it/s]
                                                                                                                                         
[2024-01-30 23:24:07] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.36.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 45%|███████████████████████████████████████████▌                                                      | 215/483 [01:03<01:59,  2.25it/s]
                                                                                                                                         
[2024-01-30 23:24:07] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.36.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 45%|███████████████████████████████████████████▌                                                      | 215/483 [01:03<01:59,  2.25it/s]
 45%|███████████████████████████████████████████▊                                                      | 216/483 [01:03<01:44,  2.54it/s]
                                                                                                                                         
[2024-01-30 23:24:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.36.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 45%|███████████████████████████████████████████▊                                                      | 216/483 [01:04<01:44,  2.54it/s]
                                                                                                                                         
[2024-01-30 23:24:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.36.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 45%|███████████████████████████████████████████▊                                                      | 216/483 [01:04<01:44,  2.54it/s]
 45%|████████████████████████████████████████████                                                      | 217/483 [01:04<02:08,  2.08it/s]
                                                                                                                                         
[2024-01-30 23:24:08] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.36.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 45%|████████████████████████████████████████████                                                      | 217/483 [01:04<02:08,  2.08it/s]
                                                                                                                                         
[2024-01-30 23:24:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.36.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 45%|████████████████████████████████████████████                                                      | 217/483 [01:04<02:08,  2.08it/s]
                                                                                                                                         
[2024-01-30 23:24:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.36.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 45%|████████████████████████████████████████████                                                      | 217/483 [01:04<02:08,  2.08it/s]
                                                                                                                                         
[2024-01-30 23:24:08] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.37.input_layernorm.weight", shape: (8192,), dtype: float32

 45%|████████████████████████████████████████████                                                      | 217/483 [01:04<02:08,  2.08it/s]
                                                                                                                                         
[2024-01-30 23:24:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.37.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 45%|████████████████████████████████████████████                                                      | 217/483 [01:04<02:08,  2.08it/s]
                                                                                                                                         
[2024-01-30 23:24:08] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.37.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 45%|████████████████████████████████████████████                                                      | 217/483 [01:04<02:08,  2.08it/s]
 46%|████████████████████████████████████████████▊                                                     | 221/483 [01:04<01:05,  3.99it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.37.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 46%|████████████████████████████████████████████▊                                                     | 221/483 [01:04<01:05,  3.99it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.37.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 46%|████████████████████████████████████████████▊                                                     | 221/483 [01:04<01:05,  3.99it/s]
 46%|█████████████████████████████████████████████                                                     | 222/483 [01:04<01:23,  3.12it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.37.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 46%|█████████████████████████████████████████████                                                     | 222/483 [01:05<01:23,  3.12it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.37.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 46%|█████████████████████████████████████████████                                                     | 222/483 [01:05<01:23,  3.12it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.37.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 46%|█████████████████████████████████████████████                                                     | 222/483 [01:05<01:23,  3.12it/s]
 46%|█████████████████████████████████████████████▍                                                    | 224/483 [01:05<01:01,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.37.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 46%|█████████████████████████████████████████████▍                                                    | 224/483 [01:05<01:01,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.37.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 46%|█████████████████████████████████████████████▍                                                    | 224/483 [01:05<01:01,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.38.input_layernorm.weight", shape: (8192,), dtype: float32

 46%|█████████████████████████████████████████████▍                                                    | 224/483 [01:05<01:01,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.38.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 46%|█████████████████████████████████████████████▍                                                    | 224/483 [01:05<01:01,  4.21it/s]
                                                                                                                                         
[2024-01-30 23:24:09] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.38.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 46%|█████████████████████████████████████████████▍                                                    | 224/483 [01:05<01:01,  4.21it/s]
 47%|██████████████████████████████████████████████                                                    | 227/483 [01:05<00:47,  5.41it/s]
                                                                                                                                         
[2024-01-30 23:24:10] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.38.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 47%|██████████████████████████████████████████████                                                    | 227/483 [01:06<00:47,  5.41it/s]
                                                                                                                                         
[2024-01-30 23:24:10] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.38.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 47%|██████████████████████████████████████████████                                                    | 227/483 [01:06<00:47,  5.41it/s]
 47%|██████████████████████████████████████████████▎                                                   | 228/483 [01:06<01:14,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:24:10] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.38.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 47%|██████████████████████████████████████████████▎                                                   | 228/483 [01:06<01:14,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:24:10] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.38.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 47%|██████████████████████████████████████████████▎                                                   | 228/483 [01:06<01:14,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:24:10] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.38.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 47%|██████████████████████████████████████████████▎                                                   | 228/483 [01:06<01:14,  3.44it/s]
 48%|██████████████████████████████████████████████▋                                                   | 230/483 [01:06<00:55,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:24:10] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.38.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 48%|██████████████████████████████████████████████▋                                                   | 230/483 [01:06<00:55,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:24:10] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.38.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 48%|██████████████████████████████████████████████▋                                                   | 230/483 [01:06<00:55,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:24:10] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00014-of-00029.bin

 48%|██████████████████████████████████████████████▋                                                   | 230/483 [01:06<00:55,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:24:11] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00015-of-00029.bin

 48%|██████████████████████████████████████████████▋                                                   | 230/483 [01:06<00:55,  4.58it/s]
                                                                                                                                         
[2024-01-30 23:24:12] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.39.input_layernorm.weight", shape: (8192,), dtype: float32

 48%|██████████████████████████████████████████████▋                                                   | 230/483 [01:08<00:55,  4.58it/s]
 48%|███████████████████████████████████████████████                                                   | 232/483 [01:08<01:41,  2.48it/s]
                                                                                                                                         
[2024-01-30 23:24:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.39.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 48%|███████████████████████████████████████████████                                                   | 232/483 [01:08<01:41,  2.48it/s]
                                                                                                                                         
[2024-01-30 23:24:12] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.39.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 48%|███████████████████████████████████████████████                                                   | 232/483 [01:08<01:41,  2.48it/s]
 48%|███████████████████████████████████████████████▎                                                  | 233/483 [01:08<01:36,  2.58it/s]
                                                                                                                                         
[2024-01-30 23:24:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.39.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 48%|███████████████████████████████████████████████▎                                                  | 233/483 [01:09<01:36,  2.58it/s]
                                                                                                                                         
[2024-01-30 23:24:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.39.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 48%|███████████████████████████████████████████████▎                                                  | 233/483 [01:09<01:36,  2.58it/s]
 48%|███████████████████████████████████████████████▍                                                  | 234/483 [01:09<01:58,  2.10it/s]
                                                                                                                                         
[2024-01-30 23:24:13] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.39.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 48%|███████████████████████████████████████████████▍                                                  | 234/483 [01:09<01:58,  2.10it/s]
                                                                                                                                         
[2024-01-30 23:24:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.39.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 48%|███████████████████████████████████████████████▍                                                  | 234/483 [01:09<01:58,  2.10it/s]
                                                                                                                                         
[2024-01-30 23:24:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.39.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 48%|███████████████████████████████████████████████▍                                                  | 234/483 [01:09<01:58,  2.10it/s]
 49%|███████████████████████████████████████████████▉                                                  | 236/483 [01:09<01:20,  3.07it/s]
                                                                                                                                         
[2024-01-30 23:24:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.39.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 49%|███████████████████████████████████████████████▉                                                  | 236/483 [01:09<01:20,  3.07it/s]
                                                                                                                                         
[2024-01-30 23:24:13] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.39.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 49%|███████████████████████████████████████████████▉                                                  | 236/483 [01:09<01:20,  3.07it/s]
                                                                                                                                         
[2024-01-30 23:24:13] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.40.input_layernorm.weight", shape: (8192,), dtype: float32

 49%|███████████████████████████████████████████████▉                                                  | 236/483 [01:09<01:20,  3.07it/s]
                                                                                                                                         
[2024-01-30 23:24:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.40.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 49%|███████████████████████████████████████████████▉                                                  | 236/483 [01:09<01:20,  3.07it/s]
                                                                                                                                         
[2024-01-30 23:24:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.40.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 49%|███████████████████████████████████████████████▉                                                  | 236/483 [01:09<01:20,  3.07it/s]
 49%|████████████████████████████████████████████████▍                                                 | 239/483 [01:09<00:56,  4.30it/s]
                                                                                                                                         
[2024-01-30 23:24:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.40.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 49%|████████████████████████████████████████████████▍                                                 | 239/483 [01:10<00:56,  4.30it/s]
                                                                                                                                         
[2024-01-30 23:24:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.40.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 49%|████████████████████████████████████████████████▍                                                 | 239/483 [01:10<00:56,  4.30it/s]
 50%|████████████████████████████████████████████████▋                                                 | 240/483 [01:10<01:18,  3.10it/s]
                                                                                                                                         
[2024-01-30 23:24:14] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.40.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 50%|████████████████████████████████████████████████▋                                                 | 240/483 [01:10<01:18,  3.10it/s]
                                                                                                                                         
[2024-01-30 23:24:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.40.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 50%|████████████████████████████████████████████████▋                                                 | 240/483 [01:10<01:18,  3.10it/s]
                                                                                                                                         
[2024-01-30 23:24:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.40.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 50%|████████████████████████████████████████████████▋                                                 | 240/483 [01:10<01:18,  3.10it/s]
 50%|█████████████████████████████████████████████████                                                 | 242/483 [01:10<00:57,  4.20it/s]
                                                                                                                                         
[2024-01-30 23:24:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.40.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 50%|█████████████████████████████████████████████████                                                 | 242/483 [01:10<00:57,  4.20it/s]
                                                                                                                                         
[2024-01-30 23:24:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.40.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 50%|█████████████████████████████████████████████████                                                 | 242/483 [01:10<00:57,  4.20it/s]
                                                                                                                                         
[2024-01-30 23:24:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.41.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 50%|█████████████████████████████████████████████████                                                 | 242/483 [01:11<00:57,  4.20it/s]
                                                                                                                                         
[2024-01-30 23:24:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.41.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 50%|█████████████████████████████████████████████████                                                 | 242/483 [01:11<00:57,  4.20it/s]
 51%|█████████████████████████████████████████████████▌                                                | 244/483 [01:11<01:12,  3.28it/s]
                                                                                                                                         
[2024-01-30 23:24:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.41.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 51%|█████████████████████████████████████████████████▌                                                | 244/483 [01:11<01:12,  3.28it/s]
                                                                                                                                         
[2024-01-30 23:24:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.41.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 51%|█████████████████████████████████████████████████▌                                                | 244/483 [01:11<01:12,  3.28it/s]
 51%|█████████████████████████████████████████████████▋                                                | 245/483 [01:11<01:04,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:16] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.41.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 51%|█████████████████████████████████████████████████▋                                                | 245/483 [01:11<01:04,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:16] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.41.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 51%|█████████████████████████████████████████████████▋                                                | 245/483 [01:11<01:04,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:16] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00015-of-00029.bin

 51%|█████████████████████████████████████████████████▋                                                | 245/483 [01:11<01:04,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:16] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00016-of-00029.bin

 51%|█████████████████████████████████████████████████▋                                                | 245/483 [01:11<01:04,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:17] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.41.input_layernorm.weight", shape: (8192,), dtype: float32

 51%|█████████████████████████████████████████████████▋                                                | 245/483 [01:13<01:04,  3.66it/s]
 51%|██████████████████████████████████████████████████                                                | 247/483 [01:13<01:51,  2.12it/s]
                                                                                                                                         
[2024-01-30 23:24:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.41.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 51%|██████████████████████████████████████████████████                                                | 247/483 [01:13<01:51,  2.12it/s]
                                                                                                                                         
[2024-01-30 23:24:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.41.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 51%|██████████████████████████████████████████████████                                                | 247/483 [01:13<01:51,  2.12it/s]
 51%|██████████████████████████████████████████████████▎                                               | 248/483 [01:13<01:43,  2.28it/s]
                                                                                                                                         
[2024-01-30 23:24:17] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.41.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 51%|██████████████████████████████████████████████████▎                                               | 248/483 [01:13<01:43,  2.28it/s]
                                                                                                                                         
[2024-01-30 23:24:17] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.42.input_layernorm.weight", shape: (8192,), dtype: float32

 51%|██████████████████████████████████████████████████▎                                               | 248/483 [01:13<01:43,  2.28it/s]
                                                                                                                                         
[2024-01-30 23:24:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.42.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 51%|██████████████████████████████████████████████████▎                                               | 248/483 [01:13<01:43,  2.28it/s]
                                                                                                                                         
[2024-01-30 23:24:18] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.42.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 51%|██████████████████████████████████████████████████▎                                               | 248/483 [01:13<01:43,  2.28it/s]
 52%|██████████████████████████████████████████████████▉                                               | 251/483 [01:13<01:03,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.42.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 52%|██████████████████████████████████████████████████▉                                               | 251/483 [01:14<01:03,  3.66it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.42.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 52%|██████████████████████████████████████████████████▉                                               | 251/483 [01:14<01:03,  3.66it/s]
 52%|███████████████████████████████████████████████████▏                                              | 252/483 [01:14<01:30,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.42.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 52%|███████████████████████████████████████████████████▏                                              | 252/483 [01:14<01:30,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.42.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 52%|███████████████████████████████████████████████████▏                                              | 252/483 [01:14<01:30,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.42.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 52%|███████████████████████████████████████████████████▏                                              | 252/483 [01:14<01:30,  2.56it/s]
 53%|███████████████████████████████████████████████████▌                                              | 254/483 [01:14<01:04,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.42.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 53%|███████████████████████████████████████████████████▌                                              | 254/483 [01:14<01:04,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.42.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 53%|███████████████████████████████████████████████████▌                                              | 254/483 [01:14<01:04,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.43.input_layernorm.weight", shape: (8192,), dtype: float32

 53%|███████████████████████████████████████████████████▌                                              | 254/483 [01:14<01:04,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.43.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 53%|███████████████████████████████████████████████████▌                                              | 254/483 [01:15<01:04,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.43.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 53%|███████████████████████████████████████████████████▌                                              | 254/483 [01:15<01:04,  3.57it/s]
 53%|████████████████████████████████████████████████████▏                                             | 257/483 [01:15<00:47,  4.74it/s]
                                                                                                                                         
[2024-01-30 23:24:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.43.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 53%|████████████████████████████████████████████████████▏                                             | 257/483 [01:16<00:47,  4.74it/s]
                                                                                                                                         
[2024-01-30 23:24:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.43.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 53%|████████████████████████████████████████████████████▏                                             | 257/483 [01:16<00:47,  4.74it/s]
 53%|████████████████████████████████████████████████████▎                                             | 258/483 [01:16<01:12,  3.12it/s]
                                                                                                                                         
[2024-01-30 23:24:20] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.43.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 53%|████████████████████████████████████████████████████▎                                             | 258/483 [01:16<01:12,  3.12it/s]
                                                                                                                                         
[2024-01-30 23:24:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.43.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 53%|████████████████████████████████████████████████████▎                                             | 258/483 [01:16<01:12,  3.12it/s]
                                                                                                                                         
[2024-01-30 23:24:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.43.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 53%|████████████████████████████████████████████████████▎                                             | 258/483 [01:16<01:12,  3.12it/s]
 54%|████████████████████████████████████████████████████▊                                             | 260/483 [01:16<00:53,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:24:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.43.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 54%|████████████████████████████████████████████████████▊                                             | 260/483 [01:16<00:53,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:24:20] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.43.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 54%|████████████████████████████████████████████████████▊                                             | 260/483 [01:16<00:53,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:24:20] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00017-of-00029.bin

 54%|████████████████████████████████████████████████████▊                                             | 260/483 [01:16<00:53,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:24:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.44.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 54%|████████████████████████████████████████████████████▊                                             | 260/483 [01:23<00:53,  4.18it/s]
                                                                                                                                         
[2024-01-30 23:24:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.44.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 54%|████████████████████████████████████████████████████▊                                             | 260/483 [01:24<00:53,  4.18it/s]
 54%|█████████████████████████████████████████████████████▏                                            | 262/483 [01:24<05:08,  1.40s/it]
                                                                                                                                         
[2024-01-30 23:24:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.44.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 54%|█████████████████████████████████████████████████████▏                                            | 262/483 [01:24<05:08,  1.40s/it]
                                                                                                                                         
[2024-01-30 23:24:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.44.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 54%|█████████████████████████████████████████████████████▏                                            | 262/483 [01:24<05:08,  1.40s/it]
 54%|█████████████████████████████████████████████████████▎                                            | 263/483 [01:24<04:18,  1.17s/it]
                                                                                                                                         
[2024-01-30 23:24:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.44.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 54%|█████████████████████████████████████████████████████▎                                            | 263/483 [01:24<04:18,  1.17s/it]
                                                                                                                                         
[2024-01-30 23:24:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.44.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 54%|█████████████████████████████████████████████████████▎                                            | 263/483 [01:24<04:18,  1.17s/it]
                                                                                                                                         
[2024-01-30 23:24:28] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.44.input_layernorm.weight", shape: (8192,), dtype: float32

 54%|█████████████████████████████████████████████████████▎                                            | 263/483 [01:24<04:18,  1.17s/it]
                                                                                                                                         
[2024-01-30 23:24:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.44.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 54%|█████████████████████████████████████████████████████▎                                            | 263/483 [01:24<04:18,  1.17s/it]
                                                                                                                                         
[2024-01-30 23:24:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.44.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 54%|█████████████████████████████████████████████████████▎                                            | 263/483 [01:24<04:18,  1.17s/it]
 55%|█████████████████████████████████████████████████████▉                                            | 266/483 [01:24<02:39,  1.36it/s]
                                                                                                                                         
[2024-01-30 23:24:29] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.44.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 55%|█████████████████████████████████████████████████████▉                                            | 266/483 [01:24<02:39,  1.36it/s]
                                                                                                                                         
[2024-01-30 23:24:29] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.45.input_layernorm.weight", shape: (8192,), dtype: float32

 55%|█████████████████████████████████████████████████████▉                                            | 266/483 [01:24<02:39,  1.36it/s]
                                                                                                                                         
[2024-01-30 23:24:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.45.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 55%|█████████████████████████████████████████████████████▉                                            | 266/483 [01:25<02:39,  1.36it/s]
                                                                                                                                         
[2024-01-30 23:24:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.45.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 55%|█████████████████████████████████████████████████████▉                                            | 266/483 [01:25<02:39,  1.36it/s]
 56%|██████████████████████████████████████████████████████▌                                           | 269/483 [01:25<01:49,  1.96it/s]
                                                                                                                                         
[2024-01-30 23:24:30] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.45.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 56%|██████████████████████████████████████████████████████▌                                           | 269/483 [01:26<01:49,  1.96it/s]
                                                                                                                                         
[2024-01-30 23:24:30] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.45.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 56%|██████████████████████████████████████████████████████▌                                           | 269/483 [01:26<01:49,  1.96it/s]
 56%|██████████████████████████████████████████████████████▊                                           | 270/483 [01:26<02:11,  1.61it/s]
                                                                                                                                         
[2024-01-30 23:24:30] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.45.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 56%|██████████████████████████████████████████████████████▊                                           | 270/483 [01:26<02:11,  1.61it/s]
                                                                                                                                         
[2024-01-30 23:24:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.45.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 56%|██████████████████████████████████████████████████████▊                                           | 270/483 [01:26<02:11,  1.61it/s]
                                                                                                                                         
[2024-01-30 23:24:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.45.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 56%|██████████████████████████████████████████████████████▊                                           | 270/483 [01:26<02:11,  1.61it/s]
 56%|███████████████████████████████████████████████████████▏                                          | 272/483 [01:26<01:37,  2.15it/s]
                                                                                                                                         
[2024-01-30 23:24:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.45.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 56%|███████████████████████████████████████████████████████▏                                          | 272/483 [01:26<01:37,  2.15it/s]
                                                                                                                                         
[2024-01-30 23:24:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.45.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 56%|███████████████████████████████████████████████████████▏                                          | 272/483 [01:26<01:37,  2.15it/s]
 57%|███████████████████████████████████████████████████████▍                                          | 273/483 [01:26<01:25,  2.45it/s]
                                                                                                                                         
[2024-01-30 23:24:31] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.46.input_layernorm.weight", shape: (8192,), dtype: float32

 57%|███████████████████████████████████████████████████████▍                                          | 273/483 [01:26<01:25,  2.45it/s]
                                                                                                                                         
[2024-01-30 23:24:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.46.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 57%|███████████████████████████████████████████████████████▍                                          | 273/483 [01:27<01:25,  2.45it/s]
                                                                                                                                         
[2024-01-30 23:24:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.46.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 57%|███████████████████████████████████████████████████████▍                                          | 273/483 [01:27<01:25,  2.45it/s]
 57%|███████████████████████████████████████████████████████▊                                          | 275/483 [01:27<01:11,  2.90it/s]
                                                                                                                                         
[2024-01-30 23:24:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.46.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 57%|███████████████████████████████████████████████████████▊                                          | 275/483 [01:28<01:11,  2.90it/s]
                                                                                                                                         
[2024-01-30 23:24:32] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.46.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 57%|███████████████████████████████████████████████████████▊                                          | 275/483 [01:28<01:11,  2.90it/s]
 57%|████████████████████████████████████████████████████████                                          | 276/483 [01:28<01:45,  1.97it/s]
                                                                                                                                         
[2024-01-30 23:24:32] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.46.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 57%|████████████████████████████████████████████████████████                                          | 276/483 [01:28<01:45,  1.97it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.46.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 57%|████████████████████████████████████████████████████████                                          | 276/483 [01:28<01:45,  1.97it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.46.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 57%|████████████████████████████████████████████████████████                                          | 276/483 [01:28<01:45,  1.97it/s]
 58%|████████████████████████████████████████████████████████▍                                         | 278/483 [01:28<01:14,  2.74it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.46.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 58%|████████████████████████████████████████████████████████▍                                         | 278/483 [01:28<01:14,  2.74it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.46.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 58%|████████████████████████████████████████████████████████▍                                         | 278/483 [01:28<01:14,  2.74it/s]
 58%|████████████████████████████████████████████████████████▌                                         | 279/483 [01:28<01:05,  3.14it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.47.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 58%|████████████████████████████████████████████████████████▌                                         | 279/483 [01:28<01:05,  3.14it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.47.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 58%|████████████████████████████████████████████████████████▌                                         | 279/483 [01:28<01:05,  3.14it/s]
 58%|████████████████████████████████████████████████████████▊                                         | 280/483 [01:28<00:57,  3.52it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.47.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 58%|████████████████████████████████████████████████████████▊                                         | 280/483 [01:29<00:57,  3.52it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.47.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 58%|████████████████████████████████████████████████████████▊                                         | 280/483 [01:29<00:57,  3.52it/s]
 58%|█████████████████████████████████████████████████████████                                         | 281/483 [01:29<00:49,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00017-of-00029.bin

 58%|█████████████████████████████████████████████████████████                                         | 281/483 [01:29<00:49,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00016-of-00029.bin

 58%|█████████████████████████████████████████████████████████                                         | 281/483 [01:29<00:49,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:24:33] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00018-of-00029.bin

 58%|█████████████████████████████████████████████████████████                                         | 281/483 [01:29<00:49,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:24:35] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.47.input_layernorm.weight", shape: (8192,), dtype: float32

 58%|█████████████████████████████████████████████████████████                                         | 281/483 [01:31<00:49,  4.09it/s]
 58%|█████████████████████████████████████████████████████████▏                                        | 282/483 [01:31<02:17,  1.46it/s]
                                                                                                                                         
[2024-01-30 23:24:35] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.47.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 58%|█████████████████████████████████████████████████████████▏                                        | 282/483 [01:31<02:17,  1.46it/s]
                                                                                                                                         
[2024-01-30 23:24:35] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.47.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 58%|█████████████████████████████████████████████████████████▏                                        | 282/483 [01:31<02:17,  1.46it/s]
 59%|█████████████████████████████████████████████████████████▍                                        | 283/483 [01:31<01:50,  1.80it/s]
                                                                                                                                         
[2024-01-30 23:24:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.47.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 59%|█████████████████████████████████████████████████████████▍                                        | 283/483 [01:31<01:50,  1.80it/s]
                                                                                                                                         
[2024-01-30 23:24:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.47.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 59%|█████████████████████████████████████████████████████████▍                                        | 283/483 [01:31<01:50,  1.80it/s]
 59%|█████████████████████████████████████████████████████████▌                                        | 284/483 [01:31<01:53,  1.75it/s]
                                                                                                                                         
[2024-01-30 23:24:36] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.47.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 59%|█████████████████████████████████████████████████████████▌                                        | 284/483 [01:31<01:53,  1.75it/s]
                                                                                                                                         
[2024-01-30 23:24:36] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.48.input_layernorm.weight", shape: (8192,), dtype: float32

 59%|█████████████████████████████████████████████████████████▌                                        | 284/483 [01:31<01:53,  1.75it/s]
                                                                                                                                         
[2024-01-30 23:24:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.48.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 59%|█████████████████████████████████████████████████████████▌                                        | 284/483 [01:32<01:53,  1.75it/s]
                                                                                                                                         
[2024-01-30 23:24:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.48.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 59%|█████████████████████████████████████████████████████████▌                                        | 284/483 [01:32<01:53,  1.75it/s]
 59%|██████████████████████████████████████████████████████████▏                                       | 287/483 [01:32<00:59,  3.31it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.48.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 59%|██████████████████████████████████████████████████████████▏                                       | 287/483 [01:32<00:59,  3.31it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.48.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 59%|██████████████████████████████████████████████████████████▏                                       | 287/483 [01:32<00:59,  3.31it/s]
 60%|██████████████████████████████████████████████████████████▍                                       | 288/483 [01:32<01:13,  2.67it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.48.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 60%|██████████████████████████████████████████████████████████▍                                       | 288/483 [01:32<01:13,  2.67it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.48.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 60%|██████████████████████████████████████████████████████████▍                                       | 288/483 [01:32<01:13,  2.67it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.48.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 60%|██████████████████████████████████████████████████████████▍                                       | 288/483 [01:32<01:13,  2.67it/s]
 60%|██████████████████████████████████████████████████████████▊                                       | 290/483 [01:32<00:50,  3.81it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.48.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 60%|██████████████████████████████████████████████████████████▊                                       | 290/483 [01:32<00:50,  3.81it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.48.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 60%|██████████████████████████████████████████████████████████▊                                       | 290/483 [01:32<00:50,  3.81it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.49.input_layernorm.weight", shape: (8192,), dtype: float32

 60%|██████████████████████████████████████████████████████████▊                                       | 290/483 [01:32<00:50,  3.81it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.49.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 60%|██████████████████████████████████████████████████████████▊                                       | 290/483 [01:33<00:50,  3.81it/s]
                                                                                                                                         
[2024-01-30 23:24:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.49.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 60%|██████████████████████████████████████████████████████████▊                                       | 290/483 [01:33<00:50,  3.81it/s]
 61%|███████████████████████████████████████████████████████████▍                                      | 293/483 [01:33<00:35,  5.35it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.49.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 61%|███████████████████████████████████████████████████████████▍                                      | 293/483 [01:33<00:35,  5.35it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.49.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 61%|███████████████████████████████████████████████████████████▍                                      | 293/483 [01:33<00:35,  5.35it/s]
 61%|███████████████████████████████████████████████████████████▋                                      | 294/483 [01:33<00:50,  3.74it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.49.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 61%|███████████████████████████████████████████████████████████▋                                      | 294/483 [01:33<00:50,  3.74it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.49.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 61%|███████████████████████████████████████████████████████████▋                                      | 294/483 [01:33<00:50,  3.74it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.49.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 61%|███████████████████████████████████████████████████████████▋                                      | 294/483 [01:33<00:50,  3.74it/s]
 61%|████████████████████████████████████████████████████████████                                      | 296/483 [01:33<00:37,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.49.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 61%|████████████████████████████████████████████████████████████                                      | 296/483 [01:34<00:37,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.49.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 61%|████████████████████████████████████████████████████████████                                      | 296/483 [01:34<00:37,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.50.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 61%|████████████████████████████████████████████████████████████                                      | 296/483 [01:34<00:37,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.50.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 61%|████████████████████████████████████████████████████████████                                      | 296/483 [01:34<00:37,  4.95it/s]
 62%|████████████████████████████████████████████████████████████▍                                     | 298/483 [01:34<00:31,  5.96it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00018-of-00029.bin

 62%|████████████████████████████████████████████████████████████▍                                     | 298/483 [01:34<00:31,  5.96it/s]
                                                                                                                                         
[2024-01-30 23:24:38] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00003-of-00029.bin

 62%|████████████████████████████████████████████████████████████▍                                     | 298/483 [01:34<00:31,  5.96it/s]
                                                                                                                                         
[2024-01-30 23:24:40] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.5.input_layernorm.weight", shape: (8192,), dtype: float32

 62%|████████████████████████████████████████████████████████████▍                                     | 298/483 [01:35<00:31,  5.96it/s]
 62%|████████████████████████████████████████████████████████████▋                                     | 299/483 [01:35<01:22,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:24:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.5.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 62%|████████████████████████████████████████████████████████████▋                                     | 299/483 [01:35<01:22,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:24:40] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.5.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 62%|████████████████████████████████████████████████████████████▋                                     | 299/483 [01:35<01:22,  2.22it/s]
 62%|████████████████████████████████████████████████████████████▊                                     | 300/483 [01:35<01:12,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:24:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.5.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 62%|████████████████████████████████████████████████████████████▊                                     | 300/483 [01:36<01:12,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:24:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.5.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 62%|████████████████████████████████████████████████████████████▊                                     | 300/483 [01:36<01:12,  2.51it/s]
 62%|█████████████████████████████████████████████████████████████                                     | 301/483 [01:36<01:28,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:24:41] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.5.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 62%|█████████████████████████████████████████████████████████████                                     | 301/483 [01:36<01:28,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:24:41] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.6.input_layernorm.weight", shape: (8192,), dtype: float32

 62%|█████████████████████████████████████████████████████████████                                     | 301/483 [01:36<01:28,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:24:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.6.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 62%|█████████████████████████████████████████████████████████████                                     | 301/483 [01:37<01:28,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:24:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.6.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 62%|█████████████████████████████████████████████████████████████                                     | 301/483 [01:37<01:28,  2.05it/s]
 63%|█████████████████████████████████████████████████████████████▋                                    | 304/483 [01:37<00:51,  3.50it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.6.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 63%|█████████████████████████████████████████████████████████████▋                                    | 304/483 [01:37<00:51,  3.50it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.6.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 63%|█████████████████████████████████████████████████████████████▋                                    | 304/483 [01:37<00:51,  3.50it/s]
 63%|█████████████████████████████████████████████████████████████▉                                    | 305/483 [01:37<01:05,  2.72it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.6.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 63%|█████████████████████████████████████████████████████████████▉                                    | 305/483 [01:37<01:05,  2.72it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.6.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 63%|█████████████████████████████████████████████████████████████▉                                    | 305/483 [01:37<01:05,  2.72it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.6.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 63%|█████████████████████████████████████████████████████████████▉                                    | 305/483 [01:37<01:05,  2.72it/s]
 64%|██████████████████████████████████████████████████████████████▎                                   | 307/483 [01:37<00:46,  3.78it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.6.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 64%|██████████████████████████████████████████████████████████████▎                                   | 307/483 [01:37<00:46,  3.78it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.6.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 64%|██████████████████████████████████████████████████████████████▎                                   | 307/483 [01:37<00:46,  3.78it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.7.input_layernorm.weight", shape: (8192,), dtype: float32

 64%|██████████████████████████████████████████████████████████████▎                                   | 307/483 [01:37<00:46,  3.78it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.7.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 64%|██████████████████████████████████████████████████████████████▎                                   | 307/483 [01:38<00:46,  3.78it/s]
                                                                                                                                         
[2024-01-30 23:24:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.7.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 64%|██████████████████████████████████████████████████████████████▎                                   | 307/483 [01:38<00:46,  3.78it/s]
 64%|██████████████████████████████████████████████████████████████▉                                   | 310/483 [01:38<00:34,  5.05it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.7.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 64%|██████████████████████████████████████████████████████████████▉                                   | 310/483 [01:38<00:34,  5.05it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.7.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 64%|██████████████████████████████████████████████████████████████▉                                   | 310/483 [01:38<00:34,  5.05it/s]
 64%|███████████████████████████████████████████████████████████████                                   | 311/483 [01:38<00:49,  3.49it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.7.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 64%|███████████████████████████████████████████████████████████████                                   | 311/483 [01:38<00:49,  3.49it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.7.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 64%|███████████████████████████████████████████████████████████████                                   | 311/483 [01:39<00:49,  3.49it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.7.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 64%|███████████████████████████████████████████████████████████████                                   | 311/483 [01:39<00:49,  3.49it/s]
 65%|███████████████████████████████████████████████████████████████▌                                  | 313/483 [01:39<00:36,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.7.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 65%|███████████████████████████████████████████████████████████████▌                                  | 313/483 [01:39<00:36,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.7.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 65%|███████████████████████████████████████████████████████████████▌                                  | 313/483 [01:39<00:36,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.8.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 65%|███████████████████████████████████████████████████████████████▌                                  | 313/483 [01:39<00:36,  4.65it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.8.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 65%|███████████████████████████████████████████████████████████████▌                                  | 313/483 [01:39<00:36,  4.65it/s]
 65%|███████████████████████████████████████████████████████████████▉                                  | 315/483 [01:39<00:28,  5.79it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00003-of-00029.bin

 65%|███████████████████████████████████████████████████████████████▉                                  | 315/483 [01:39<00:28,  5.79it/s]
                                                                                                                                         
[2024-01-30 23:24:43] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00019-of-00029.bin

 65%|███████████████████████████████████████████████████████████████▉                                  | 315/483 [01:39<00:28,  5.79it/s]
                                                                                                                                         
[2024-01-30 23:24:45] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.50.input_layernorm.weight", shape: (8192,), dtype: float32

 65%|███████████████████████████████████████████████████████████████▉                                  | 315/483 [01:40<00:28,  5.79it/s]
 65%|████████████████████████████████████████████████████████████████                                  | 316/483 [01:40<01:15,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:24:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.50.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 65%|████████████████████████████████████████████████████████████████                                  | 316/483 [01:41<01:15,  2.22it/s]
                                                                                                                                         
[2024-01-30 23:24:45] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.50.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 65%|████████████████████████████████████████████████████████████████                                  | 316/483 [01:41<01:15,  2.22it/s]
 66%|████████████████████████████████████████████████████████████████▎                                 | 317/483 [01:41<01:06,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:24:47] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.50.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 66%|████████████████████████████████████████████████████████████████▎                                 | 317/483 [01:43<01:06,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:24:47] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.50.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 66%|████████████████████████████████████████████████████████████████▎                                 | 317/483 [01:43<01:06,  2.51it/s]
 66%|████████████████████████████████████████████████████████████████▌                                 | 318/483 [01:43<02:06,  1.30it/s]
                                                                                                                                         
[2024-01-30 23:24:47] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.50.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 66%|████████████████████████████████████████████████████████████████▌                                 | 318/483 [01:43<02:06,  1.30it/s]
                                                                                                                                         
[2024-01-30 23:24:47] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.50.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 66%|████████████████████████████████████████████████████████████████▌                                 | 318/483 [01:43<02:06,  1.30it/s]
                                                                                                                                         
[2024-01-30 23:24:47] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.50.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 66%|████████████████████████████████████████████████████████████████▌                                 | 318/483 [01:43<02:06,  1.30it/s]
                                                                                                                                         
[2024-01-30 23:24:47] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.51.input_layernorm.weight", shape: (8192,), dtype: float32

 66%|████████████████████████████████████████████████████████████████▌                                 | 318/483 [01:43<02:06,  1.30it/s]
 66%|█████████████████████████████████████████████████████████████████▏                                | 321/483 [01:43<01:05,  2.48it/s]
                                                                                                                                         
[2024-01-30 23:24:47] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.51.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 66%|█████████████████████████████████████████████████████████████████▏                                | 321/483 [01:43<01:05,  2.48it/s]
                                                                                                                                         
[2024-01-30 23:24:47] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.51.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 66%|█████████████████████████████████████████████████████████████████▏                                | 321/483 [01:43<01:05,  2.48it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.51.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 66%|█████████████████████████████████████████████████████████████████▏                                | 321/483 [01:44<01:05,  2.48it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.51.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 66%|█████████████████████████████████████████████████████████████████▏                                | 321/483 [01:44<01:05,  2.48it/s]
 67%|█████████████████████████████████████████████████████████████████▌                                | 323/483 [01:44<01:26,  1.86it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.51.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 67%|█████████████████████████████████████████████████████████████████▌                                | 323/483 [01:44<01:26,  1.86it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.51.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 67%|█████████████████████████████████████████████████████████████████▌                                | 323/483 [01:44<01:26,  1.86it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.51.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 67%|█████████████████████████████████████████████████████████████████▌                                | 323/483 [01:44<01:26,  1.86it/s]
 67%|█████████████████████████████████████████████████████████████████▉                                | 325/483 [01:45<01:01,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.51.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 67%|█████████████████████████████████████████████████████████████████▉                                | 325/483 [01:45<01:01,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.51.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 67%|█████████████████████████████████████████████████████████████████▉                                | 325/483 [01:45<01:01,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.52.input_layernorm.weight", shape: (8192,), dtype: float32

 67%|█████████████████████████████████████████████████████████████████▉                                | 325/483 [01:45<01:01,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.52.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 67%|█████████████████████████████████████████████████████████████████▉                                | 325/483 [01:45<01:01,  2.56it/s]
                                                                                                                                         
[2024-01-30 23:24:49] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.52.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 67%|█████████████████████████████████████████████████████████████████▉                                | 325/483 [01:45<01:01,  2.56it/s]
 68%|██████████████████████████████████████████████████████████████████▌                               | 328/483 [01:45<00:43,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.52.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 68%|██████████████████████████████████████████████████████████████████▌                               | 328/483 [01:46<00:43,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.52.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 68%|██████████████████████████████████████████████████████████████████▌                               | 328/483 [01:46<00:43,  3.57it/s]
 68%|██████████████████████████████████████████████████████████████████▊                               | 329/483 [01:46<00:58,  2.65it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.52.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 68%|██████████████████████████████████████████████████████████████████▊                               | 329/483 [01:46<00:58,  2.65it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.52.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 68%|██████████████████████████████████████████████████████████████████▊                               | 329/483 [01:46<00:58,  2.65it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.52.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 68%|██████████████████████████████████████████████████████████████████▊                               | 329/483 [01:46<00:58,  2.65it/s]
 69%|███████████████████████████████████████████████████████████████████▏                              | 331/483 [01:46<00:42,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.52.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 69%|███████████████████████████████████████████████████████████████████▏                              | 331/483 [01:46<00:42,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.52.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 69%|███████████████████████████████████████████████████████████████████▏                              | 331/483 [01:46<00:42,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00019-of-00029.bin

 69%|███████████████████████████████████████████████████████████████████▏                              | 331/483 [01:46<00:42,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:50] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00020-of-00029.bin

 69%|███████████████████████████████████████████████████████████████████▏                              | 331/483 [01:46<00:42,  3.57it/s]
                                                                                                                                         
[2024-01-30 23:24:52] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.53.input_layernorm.weight", shape: (8192,), dtype: float32

 69%|███████████████████████████████████████████████████████████████████▏                              | 331/483 [01:48<00:42,  3.57it/s]
 69%|███████████████████████████████████████████████████████████████████▌                              | 333/483 [01:48<01:07,  2.21it/s]
                                                                                                                                         
[2024-01-30 23:24:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.53.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 69%|███████████████████████████████████████████████████████████████████▌                              | 333/483 [01:48<01:07,  2.21it/s]
                                                                                                                                         
[2024-01-30 23:24:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.53.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 69%|███████████████████████████████████████████████████████████████████▌                              | 333/483 [01:48<01:07,  2.21it/s]
 69%|███████████████████████████████████████████████████████████████████▊                              | 334/483 [01:48<01:02,  2.37it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.53.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 69%|███████████████████████████████████████████████████████████████████▊                              | 334/483 [01:49<01:02,  2.37it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.53.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 69%|███████████████████████████████████████████████████████████████████▊                              | 334/483 [01:49<01:02,  2.37it/s]
 69%|███████████████████████████████████████████████████████████████████▉                              | 335/483 [01:49<01:38,  1.50it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.53.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 69%|███████████████████████████████████████████████████████████████████▉                              | 335/483 [01:49<01:38,  1.50it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.53.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 69%|███████████████████████████████████████████████████████████████████▉                              | 335/483 [01:50<01:38,  1.50it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.53.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 69%|███████████████████████████████████████████████████████████████████▉                              | 335/483 [01:50<01:38,  1.50it/s]
 70%|████████████████████████████████████████████████████████████████████▍                             | 337/483 [01:50<01:05,  2.24it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.53.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 70%|████████████████████████████████████████████████████████████████████▍                             | 337/483 [01:50<01:05,  2.24it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.53.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 70%|████████████████████████████████████████████████████████████████████▍                             | 337/483 [01:50<01:05,  2.24it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.54.input_layernorm.weight", shape: (8192,), dtype: float32

 70%|████████████████████████████████████████████████████████████████████▍                             | 337/483 [01:50<01:05,  2.24it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.54.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 70%|████████████████████████████████████████████████████████████████████▍                             | 337/483 [01:50<01:05,  2.24it/s]
                                                                                                                                         
[2024-01-30 23:24:54] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.54.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 70%|████████████████████████████████████████████████████████████████████▍                             | 337/483 [01:50<01:05,  2.24it/s]
 70%|████████████████████████████████████████████████████████████████████▉                             | 340/483 [01:50<00:42,  3.34it/s]
                                                                                                                                         
[2024-01-30 23:24:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.54.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 70%|████████████████████████████████████████████████████████████████████▉                             | 340/483 [01:51<00:42,  3.34it/s]
                                                                                                                                         
[2024-01-30 23:24:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.54.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 70%|████████████████████████████████████████████████████████████████████▉                             | 340/483 [01:51<00:42,  3.34it/s]
 71%|█████████████████████████████████████████████████████████████████████▏                            | 341/483 [01:51<00:56,  2.50it/s]
                                                                                                                                         
[2024-01-30 23:24:55] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.54.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 71%|█████████████████████████████████████████████████████████████████████▏                            | 341/483 [01:51<00:56,  2.50it/s]
                                                                                                                                         
[2024-01-30 23:24:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.54.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 71%|█████████████████████████████████████████████████████████████████████▏                            | 341/483 [01:51<00:56,  2.50it/s]
                                                                                                                                         
[2024-01-30 23:24:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.54.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 71%|█████████████████████████████████████████████████████████████████████▏                            | 341/483 [01:51<00:56,  2.50it/s]
 71%|█████████████████████████████████████████████████████████████████████▌                            | 343/483 [01:51<00:41,  3.41it/s]
                                                                                                                                         
[2024-01-30 23:24:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.54.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 71%|█████████████████████████████████████████████████████████████████████▌                            | 343/483 [01:51<00:41,  3.41it/s]
                                                                                                                                         
[2024-01-30 23:24:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.54.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 71%|█████████████████████████████████████████████████████████████████████▌                            | 343/483 [01:51<00:41,  3.41it/s]
                                                                                                                                         
[2024-01-30 23:24:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.55.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 71%|█████████████████████████████████████████████████████████████████████▌                            | 343/483 [01:52<00:41,  3.41it/s]
                                                                                                                                         
[2024-01-30 23:24:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.55.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 71%|█████████████████████████████████████████████████████████████████████▌                            | 343/483 [01:52<00:41,  3.41it/s]
 71%|██████████████████████████████████████████████████████████████████████                            | 345/483 [01:52<00:47,  2.93it/s]
                                                                                                                                         
[2024-01-30 23:24:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.55.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 71%|██████████████████████████████████████████████████████████████████████                            | 345/483 [01:52<00:47,  2.93it/s]
                                                                                                                                         
[2024-01-30 23:24:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.55.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 71%|██████████████████████████████████████████████████████████████████████                            | 345/483 [01:52<00:47,  2.93it/s]
 72%|██████████████████████████████████████████████████████████████████████▏                           | 346/483 [01:52<00:41,  3.32it/s]
                                                                                                                                         
[2024-01-30 23:24:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.55.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 72%|██████████████████████████████████████████████████████████████████████▏                           | 346/483 [01:52<00:41,  3.32it/s]
                                                                                                                                         
[2024-01-30 23:24:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.55.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 72%|██████████████████████████████████████████████████████████████████████▏                           | 346/483 [01:52<00:41,  3.32it/s]
 72%|██████████████████████████████████████████████████████████████████████▍                           | 347/483 [01:52<00:35,  3.86it/s]
                                                                                                                                         
[2024-01-30 23:24:56] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00020-of-00029.bin

 72%|██████████████████████████████████████████████████████████████████████▍                           | 347/483 [01:52<00:35,  3.86it/s]
                                                                                                                                         
[2024-01-30 23:24:57] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00021-of-00029.bin

 72%|██████████████████████████████████████████████████████████████████████▍                           | 347/483 [01:52<00:35,  3.86it/s]
                                                                                                                                         
[2024-01-30 23:24:58] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.55.input_layernorm.weight", shape: (8192,), dtype: float32

 72%|██████████████████████████████████████████████████████████████████████▍                           | 347/483 [01:54<00:35,  3.86it/s]
 72%|██████████████████████████████████████████████████████████████████████▌                           | 348/483 [01:54<01:18,  1.73it/s]
                                                                                                                                         
[2024-01-30 23:24:58] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.55.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 72%|██████████████████████████████████████████████████████████████████████▌                           | 348/483 [01:54<01:18,  1.73it/s]
                                                                                                                                         
[2024-01-30 23:24:58] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.55.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 72%|██████████████████████████████████████████████████████████████████████▌                           | 348/483 [01:54<01:18,  1.73it/s]
 72%|██████████████████████████████████████████████████████████████████████▊                           | 349/483 [01:54<01:04,  2.06it/s]
                                                                                                                                         
[2024-01-30 23:24:58] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.55.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 72%|██████████████████████████████████████████████████████████████████████▊                           | 349/483 [01:54<01:04,  2.06it/s]
                                                                                                                                         
[2024-01-30 23:24:58] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.56.input_layernorm.weight", shape: (8192,), dtype: float32

 72%|██████████████████████████████████████████████████████████████████████▊                           | 349/483 [01:54<01:04,  2.06it/s]
                                                                                                                                         
[2024-01-30 23:24:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.56.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 72%|██████████████████████████████████████████████████████████████████████▊                           | 349/483 [01:54<01:04,  2.06it/s]
                                                                                                                                         
[2024-01-30 23:24:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.56.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 72%|██████████████████████████████████████████████████████████████████████▊                           | 349/483 [01:54<01:04,  2.06it/s]
 73%|███████████████████████████████████████████████████████████████████████▍                          | 352/483 [01:54<00:35,  3.69it/s]
                                                                                                                                         
[2024-01-30 23:25:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.56.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 73%|███████████████████████████████████████████████████████████████████████▍                          | 352/483 [01:57<00:35,  3.69it/s]
                                                                                                                                         
[2024-01-30 23:25:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.56.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 73%|███████████████████████████████████████████████████████████████████████▍                          | 352/483 [01:57<00:35,  3.69it/s]
 73%|███████████████████████████████████████████████████████████████████████▌                          | 353/483 [01:57<01:34,  1.38it/s]
                                                                                                                                         
[2024-01-30 23:25:01] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.56.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 73%|███████████████████████████████████████████████████████████████████████▌                          | 353/483 [01:57<01:34,  1.38it/s]
                                                                                                                                         
[2024-01-30 23:25:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.56.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 73%|███████████████████████████████████████████████████████████████████████▌                          | 353/483 [01:57<01:34,  1.38it/s]
                                                                                                                                         
[2024-01-30 23:25:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.56.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 73%|███████████████████████████████████████████████████████████████████████▌                          | 353/483 [01:57<01:34,  1.38it/s]
 73%|████████████████████████████████████████████████████████████████████████                          | 355/483 [01:57<01:02,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:25:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.56.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 73%|████████████████████████████████████████████████████████████████████████                          | 355/483 [01:57<01:02,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:25:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.56.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 73%|████████████████████████████████████████████████████████████████████████                          | 355/483 [01:57<01:02,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:25:01] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.57.input_layernorm.weight", shape: (8192,), dtype: float32

 73%|████████████████████████████████████████████████████████████████████████                          | 355/483 [01:57<01:02,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:25:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.57.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 73%|████████████████████████████████████████████████████████████████████████                          | 355/483 [01:57<01:02,  2.05it/s]
                                                                                                                                         
[2024-01-30 23:25:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.57.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 73%|████████████████████████████████████████████████████████████████████████                          | 355/483 [01:57<01:02,  2.05it/s]
 74%|████████████████████████████████████████████████████████████████████████▋                         | 358/483 [01:57<00:40,  3.07it/s]
                                                                                                                                         
[2024-01-30 23:25:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.57.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 74%|████████████████████████████████████████████████████████████████████████▋                         | 358/483 [01:58<00:40,  3.07it/s]
                                                                                                                                         
[2024-01-30 23:25:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.57.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 74%|████████████████████████████████████████████████████████████████████████▋                         | 358/483 [01:58<00:40,  3.07it/s]
 74%|████████████████████████████████████████████████████████████████████████▊                         | 359/483 [01:58<00:52,  2.37it/s]
                                                                                                                                         
[2024-01-30 23:25:03] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.57.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 74%|████████████████████████████████████████████████████████████████████████▊                         | 359/483 [01:58<00:52,  2.37it/s]
                                                                                                                                         
[2024-01-30 23:25:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.57.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 74%|████████████████████████████████████████████████████████████████████████▊                         | 359/483 [01:58<00:52,  2.37it/s]
                                                                                                                                         
[2024-01-30 23:25:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.57.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 74%|████████████████████████████████████████████████████████████████████████▊                         | 359/483 [01:58<00:52,  2.37it/s]
 75%|█████████████████████████████████████████████████████████████████████████▏                        | 361/483 [01:58<00:37,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.57.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 75%|█████████████████████████████████████████████████████████████████████████▏                        | 361/483 [01:58<00:37,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.57.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 75%|█████████████████████████████████████████████████████████████████████████▏                        | 361/483 [01:58<00:37,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:03] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00022-of-00029.bin

 75%|█████████████████████████████████████████████████████████████████████████▏                        | 361/483 [01:58<00:37,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.58.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 75%|█████████████████████████████████████████████████████████████████████████▏                        | 361/483 [02:10<00:37,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.58.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 75%|█████████████████████████████████████████████████████████████████████████▏                        | 361/483 [02:10<00:37,  3.26it/s]
 75%|█████████████████████████████████████████████████████████████████████████▋                        | 363/483 [02:10<04:02,  2.02s/it]
                                                                                                                                         
[2024-01-30 23:25:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.58.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 75%|█████████████████████████████████████████████████████████████████████████▋                        | 363/483 [02:10<04:02,  2.02s/it]
                                                                                                                                         
[2024-01-30 23:25:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.58.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 75%|█████████████████████████████████████████████████████████████████████████▋                        | 363/483 [02:10<04:02,  2.02s/it]
 75%|█████████████████████████████████████████████████████████████████████████▊                        | 364/483 [02:10<03:19,  1.68s/it]
                                                                                                                                         
[2024-01-30 23:25:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.58.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 75%|█████████████████████████████████████████████████████████████████████████▊                        | 364/483 [02:10<03:19,  1.68s/it]
                                                                                                                                         
[2024-01-30 23:25:14] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.58.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 75%|█████████████████████████████████████████████████████████████████████████▊                        | 364/483 [02:10<03:19,  1.68s/it]
 76%|██████████████████████████████████████████████████████████████████████████                        | 365/483 [02:10<02:40,  1.36s/it]
                                                                                                                                         
[2024-01-30 23:25:14] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.58.input_layernorm.weight", shape: (8192,), dtype: float32

 76%|██████████████████████████████████████████████████████████████████████████                        | 365/483 [02:10<02:40,  1.36s/it]
                                                                                                                                         
[2024-01-30 23:25:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.58.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 76%|██████████████████████████████████████████████████████████████████████████                        | 365/483 [02:10<02:40,  1.36s/it]
                                                                                                                                         
[2024-01-30 23:25:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.58.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 76%|██████████████████████████████████████████████████████████████████████████                        | 365/483 [02:10<02:40,  1.36s/it]
 76%|██████████████████████████████████████████████████████████████████████████▍                       | 367/483 [02:10<01:49,  1.06it/s]
                                                                                                                                         
[2024-01-30 23:25:15] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.58.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 76%|██████████████████████████████████████████████████████████████████████████▍                       | 367/483 [02:10<01:49,  1.06it/s]
                                                                                                                                         
[2024-01-30 23:25:15] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.59.input_layernorm.weight", shape: (8192,), dtype: float32

 76%|██████████████████████████████████████████████████████████████████████████▍                       | 367/483 [02:10<01:49,  1.06it/s]
                                                                                                                                         
[2024-01-30 23:25:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.59.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 76%|██████████████████████████████████████████████████████████████████████████▍                       | 367/483 [02:11<01:49,  1.06it/s]
                                                                                                                                         
[2024-01-30 23:25:15] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.59.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 76%|██████████████████████████████████████████████████████████████████████████▍                       | 367/483 [02:11<01:49,  1.06it/s]
 77%|███████████████████████████████████████████████████████████████████████████                       | 370/483 [02:11<01:07,  1.66it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.59.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 77%|███████████████████████████████████████████████████████████████████████████                       | 370/483 [02:12<01:07,  1.66it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.59.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 77%|███████████████████████████████████████████████████████████████████████████                       | 370/483 [02:12<01:07,  1.66it/s]
 77%|███████████████████████████████████████████████████████████████████████████▎                      | 371/483 [02:12<01:21,  1.38it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.59.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 77%|███████████████████████████████████████████████████████████████████████████▎                      | 371/483 [02:12<01:21,  1.38it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.59.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 77%|███████████████████████████████████████████████████████████████████████████▎                      | 371/483 [02:12<01:21,  1.38it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.59.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 77%|███████████████████████████████████████████████████████████████████████████▎                      | 371/483 [02:12<01:21,  1.38it/s]
 77%|███████████████████████████████████████████████████████████████████████████▋                      | 373/483 [02:12<00:57,  1.91it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.59.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 77%|███████████████████████████████████████████████████████████████████████████▋                      | 373/483 [02:13<00:57,  1.91it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.59.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 77%|███████████████████████████████████████████████████████████████████████████▋                      | 373/483 [02:13<00:57,  1.91it/s]
 77%|███████████████████████████████████████████████████████████████████████████▉                      | 374/483 [02:13<00:48,  2.24it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.60.input_layernorm.weight", shape: (8192,), dtype: float32

 77%|███████████████████████████████████████████████████████████████████████████▉                      | 374/483 [02:13<00:48,  2.24it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.60.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 77%|███████████████████████████████████████████████████████████████████████████▉                      | 374/483 [02:13<00:48,  2.24it/s]
                                                                                                                                         
[2024-01-30 23:25:17] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.60.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 77%|███████████████████████████████████████████████████████████████████████████▉                      | 374/483 [02:13<00:48,  2.24it/s]
 78%|████████████████████████████████████████████████████████████████████████████▎                     | 376/483 [02:13<00:39,  2.69it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.60.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 78%|████████████████████████████████████████████████████████████████████████████▎                     | 376/483 [02:14<00:39,  2.69it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.60.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 78%|████████████████████████████████████████████████████████████████████████████▎                     | 376/483 [02:14<00:39,  2.69it/s]
 78%|████████████████████████████████████████████████████████████████████████████▍                     | 377/483 [02:14<00:58,  1.83it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.60.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 78%|████████████████████████████████████████████████████████████████████████████▍                     | 377/483 [02:14<00:58,  1.83it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.60.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 78%|████████████████████████████████████████████████████████████████████████████▍                     | 377/483 [02:14<00:58,  1.83it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.60.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 78%|████████████████████████████████████████████████████████████████████████████▍                     | 377/483 [02:15<00:58,  1.83it/s]
 78%|████████████████████████████████████████████████████████████████████████████▉                     | 379/483 [02:15<00:40,  2.58it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.60.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 78%|████████████████████████████████████████████████████████████████████████████▉                     | 379/483 [02:15<00:40,  2.58it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.60.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 78%|████████████████████████████████████████████████████████████████████████████▉                     | 379/483 [02:15<00:40,  2.58it/s]
 79%|█████████████████████████████████████████████████████████████████████████████                     | 380/483 [02:15<00:34,  2.99it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.61.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 79%|█████████████████████████████████████████████████████████████████████████████                     | 380/483 [02:15<00:34,  2.99it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.61.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 79%|█████████████████████████████████████████████████████████████████████████████                     | 380/483 [02:15<00:34,  2.99it/s]
 79%|█████████████████████████████████████████████████████████████████████████████▎                    | 381/483 [02:15<00:29,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.61.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 79%|█████████████████████████████████████████████████████████████████████████████▎                    | 381/483 [02:15<00:29,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.61.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 79%|█████████████████████████████████████████████████████████████████████████████▎                    | 381/483 [02:15<00:29,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:25:19] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00021-of-00029.bin

 79%|█████████████████████████████████████████████████████████████████████████████▎                    | 381/483 [02:15<00:29,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:25:20] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00022-of-00029.bin

 79%|█████████████████████████████████████████████████████████████████████████████▎                    | 381/483 [02:15<00:29,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:25:20] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00023-of-00029.bin

 79%|█████████████████████████████████████████████████████████████████████████████▎                    | 381/483 [02:15<00:29,  3.44it/s]
                                                                                                                                         
[2024-01-30 23:25:21] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.61.input_layernorm.weight", shape: (8192,), dtype: float32

 79%|█████████████████████████████████████████████████████████████████████████████▎                    | 381/483 [02:17<00:29,  3.44it/s]
 79%|█████████████████████████████████████████████████████████████████████████████▋                    | 383/483 [02:17<01:01,  1.63it/s]
                                                                                                                                         
[2024-01-30 23:25:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.61.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 79%|█████████████████████████████████████████████████████████████████████████████▋                    | 383/483 [02:17<01:01,  1.63it/s]
                                                                                                                                         
[2024-01-30 23:25:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.61.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 79%|█████████████████████████████████████████████████████████████████████████████▋                    | 383/483 [02:17<01:01,  1.63it/s]
 80%|█████████████████████████████████████████████████████████████████████████████▉                    | 384/483 [02:17<00:51,  1.92it/s]
                                                                                                                                         
[2024-01-30 23:25:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.61.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 80%|█████████████████████████████████████████████████████████████████████████████▉                    | 384/483 [02:18<00:51,  1.92it/s]
                                                                                                                                         
[2024-01-30 23:25:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.61.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 80%|█████████████████████████████████████████████████████████████████████████████▉                    | 384/483 [02:18<00:51,  1.92it/s]
 80%|██████████████████████████████████████████████████████████████████████████████                    | 385/483 [02:18<00:51,  1.89it/s]
                                                                                                                                         
[2024-01-30 23:25:22] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.61.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 80%|██████████████████████████████████████████████████████████████████████████████                    | 385/483 [02:18<00:51,  1.89it/s]
                                                                                                                                         
[2024-01-30 23:25:22] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.62.input_layernorm.weight", shape: (8192,), dtype: float32

 80%|██████████████████████████████████████████████████████████████████████████████                    | 385/483 [02:18<00:51,  1.89it/s]
                                                                                                                                         
[2024-01-30 23:25:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.62.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 80%|██████████████████████████████████████████████████████████████████████████████                    | 385/483 [02:18<00:51,  1.89it/s]
                                                                                                                                         
[2024-01-30 23:25:22] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.62.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 80%|██████████████████████████████████████████████████████████████████████████████                    | 385/483 [02:18<00:51,  1.89it/s]
 80%|██████████████████████████████████████████████████████████████████████████████▋                   | 388/483 [02:18<00:28,  3.34it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.62.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 80%|██████████████████████████████████████████████████████████████████████████████▋                   | 388/483 [02:18<00:28,  3.34it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.62.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 80%|██████████████████████████████████████████████████████████████████████████████▋                   | 388/483 [02:19<00:28,  3.34it/s]
 81%|██████████████████████████████████████████████████████████████████████████████▉                   | 389/483 [02:19<00:32,  2.87it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.62.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 81%|██████████████████████████████████████████████████████████████████████████████▉                   | 389/483 [02:19<00:32,  2.87it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.62.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 81%|██████████████████████████████████████████████████████████████████████████████▉                   | 389/483 [02:19<00:32,  2.87it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.62.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 81%|██████████████████████████████████████████████████████████████████████████████▉                   | 389/483 [02:19<00:32,  2.87it/s]
 81%|███████████████████████████████████████████████████████████████████████████████▎                  | 391/483 [02:19<00:22,  4.00it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.62.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 81%|███████████████████████████████████████████████████████████████████████████████▎                  | 391/483 [02:19<00:22,  4.00it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.62.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 81%|███████████████████████████████████████████████████████████████████████████████▎                  | 391/483 [02:19<00:22,  4.00it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.63.input_layernorm.weight", shape: (8192,), dtype: float32

 81%|███████████████████████████████████████████████████████████████████████████████▎                  | 391/483 [02:19<00:22,  4.00it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.63.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 81%|███████████████████████████████████████████████████████████████████████████████▎                  | 391/483 [02:19<00:22,  4.00it/s]
                                                                                                                                         
[2024-01-30 23:25:23] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.63.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 81%|███████████████████████████████████████████████████████████████████████████████▎                  | 391/483 [02:19<00:22,  4.00it/s]
 82%|███████████████████████████████████████████████████████████████████████████████▉                  | 394/483 [02:19<00:15,  5.64it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.63.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 82%|███████████████████████████████████████████████████████████████████████████████▉                  | 394/483 [02:19<00:15,  5.64it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.63.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 82%|███████████████████████████████████████████████████████████████████████████████▉                  | 394/483 [02:19<00:15,  5.64it/s]
 82%|████████████████████████████████████████████████████████████████████████████████▏                 | 395/483 [02:19<00:21,  4.14it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.63.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 82%|████████████████████████████████████████████████████████████████████████████████▏                 | 395/483 [02:19<00:21,  4.14it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.63.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 82%|████████████████████████████████████████████████████████████████████████████████▏                 | 395/483 [02:20<00:21,  4.14it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.63.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 82%|████████████████████████████████████████████████████████████████████████████████▏                 | 395/483 [02:20<00:21,  4.14it/s]
 82%|████████████████████████████████████████████████████████████████████████████████▌                 | 397/483 [02:20<00:16,  5.35it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.63.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 82%|████████████████████████████████████████████████████████████████████████████████▌                 | 397/483 [02:20<00:16,  5.35it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.63.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 82%|████████████████████████████████████████████████████████████████████████████████▌                 | 397/483 [02:20<00:16,  5.35it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.64.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 82%|████████████████████████████████████████████████████████████████████████████████▌                 | 397/483 [02:20<00:16,  5.35it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.64.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 82%|████████████████████████████████████████████████████████████████████████████████▌                 | 397/483 [02:20<00:16,  5.35it/s]
 83%|████████████████████████████████████████████████████████████████████████████████▉                 | 399/483 [02:20<00:12,  6.57it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00023-of-00029.bin

 83%|████████████████████████████████████████████████████████████████████████████████▉                 | 399/483 [02:20<00:12,  6.57it/s]
                                                                                                                                         
[2024-01-30 23:25:24] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00024-of-00029.bin

 83%|████████████████████████████████████████████████████████████████████████████████▉                 | 399/483 [02:20<00:12,  6.57it/s]
                                                                                                                                         
[2024-01-30 23:25:26] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.64.input_layernorm.weight", shape: (8192,), dtype: float32

 83%|████████████████████████████████████████████████████████████████████████████████▉                 | 399/483 [02:21<00:12,  6.57it/s]
                                                                                                                                         
[2024-01-30 23:25:26] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.64.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 83%|████████████████████████████████████████████████████████████████████████████████▉                 | 399/483 [02:22<00:12,  6.57it/s]
                                                                                                                                         
[2024-01-30 23:25:26] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.64.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 83%|████████████████████████████████████████████████████████████████████████████████▉                 | 399/483 [02:22<00:12,  6.57it/s]
 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 401/483 [02:22<00:32,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:25:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.64.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 401/483 [02:22<00:32,  2.51it/s]
                                                                                                                                         
[2024-01-30 23:25:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.64.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 401/483 [02:22<00:32,  2.51it/s]
 83%|█████████████████████████████████████████████████████████████████████████████████▌                | 402/483 [02:22<00:36,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:25:27] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.64.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 83%|█████████████████████████████████████████████████████████████████████████████████▌                | 402/483 [02:22<00:36,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:25:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.64.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 83%|█████████████████████████████████████████████████████████████████████████████████▌                | 402/483 [02:22<00:36,  2.20it/s]
                                                                                                                                         
[2024-01-30 23:25:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.64.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 83%|█████████████████████████████████████████████████████████████████████████████████▌                | 402/483 [02:22<00:36,  2.20it/s]
 84%|█████████████████████████████████████████████████████████████████████████████████▉                | 404/483 [02:22<00:24,  3.16it/s]
                                                                                                                                         
[2024-01-30 23:25:27] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.65.input_layernorm.weight", shape: (8192,), dtype: float32

 84%|█████████████████████████████████████████████████████████████████████████████████▉                | 404/483 [02:22<00:24,  3.16it/s]
                                                                                                                                         
[2024-01-30 23:25:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.65.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 84%|█████████████████████████████████████████████████████████████████████████████████▉                | 404/483 [02:23<00:24,  3.16it/s]
                                                                                                                                         
[2024-01-30 23:25:27] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.65.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 84%|█████████████████████████████████████████████████████████████████████████████████▉                | 404/483 [02:23<00:24,  3.16it/s]
 84%|██████████████████████████████████████████████████████████████████████████████████▍               | 406/483 [02:23<00:18,  4.13it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.65.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 84%|██████████████████████████████████████████████████████████████████████████████████▍               | 406/483 [02:23<00:18,  4.13it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.65.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 84%|██████████████████████████████████████████████████████████████████████████████████▍               | 406/483 [02:23<00:18,  4.13it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.65.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 84%|██████████████████████████████████████████████████████████████████████████████████▍               | 406/483 [02:23<00:18,  4.13it/s]
 84%|██████████████████████████████████████████████████████████████████████████████████▊               | 408/483 [02:23<00:20,  3.69it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.65.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 84%|██████████████████████████████████████████████████████████████████████████████████▊               | 408/483 [02:23<00:20,  3.69it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.65.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 84%|██████████████████████████████████████████████████████████████████████████████████▊               | 408/483 [02:23<00:20,  3.69it/s]
 85%|██████████████████████████████████████████████████████████████████████████████████▉               | 409/483 [02:23<00:17,  4.14it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.65.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 85%|██████████████████████████████████████████████████████████████████████████████████▉               | 409/483 [02:23<00:17,  4.14it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.65.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 85%|██████████████████████████████████████████████████████████████████████████████████▉               | 409/483 [02:23<00:17,  4.14it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.66.input_layernorm.weight", shape: (8192,), dtype: float32

 85%|██████████████████████████████████████████████████████████████████████████████████▉               | 409/483 [02:23<00:17,  4.14it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.66.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 85%|██████████████████████████████████████████████████████████████████████████████████▉               | 409/483 [02:24<00:17,  4.14it/s]
                                                                                                                                         
[2024-01-30 23:25:28] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.66.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 85%|██████████████████████████████████████████████████████████████████████████████████▉               | 409/483 [02:24<00:17,  4.14it/s]
 85%|███████████████████████████████████████████████████████████████████████████████████▌              | 412/483 [02:24<00:12,  5.83it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.66.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 85%|███████████████████████████████████████████████████████████████████████████████████▌              | 412/483 [02:24<00:12,  5.83it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.66.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 85%|███████████████████████████████████████████████████████████████████████████████████▌              | 412/483 [02:24<00:12,  5.83it/s]
 86%|███████████████████████████████████████████████████████████████████████████████████▊              | 413/483 [02:24<00:17,  3.93it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.66.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 86%|███████████████████████████████████████████████████████████████████████████████████▊              | 413/483 [02:24<00:17,  3.93it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.66.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 86%|███████████████████████████████████████████████████████████████████████████████████▊              | 413/483 [02:24<00:17,  3.93it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.66.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 86%|███████████████████████████████████████████████████████████████████████████████████▊              | 413/483 [02:24<00:17,  3.93it/s]
 86%|████████████████████████████████████████████████████████████████████████████████████▏             | 415/483 [02:24<00:13,  5.16it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.66.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 86%|████████████████████████████████████████████████████████████████████████████████████▏             | 415/483 [02:25<00:13,  5.16it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.66.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 86%|████████████████████████████████████████████████████████████████████████████████████▏             | 415/483 [02:25<00:13,  5.16it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00024-of-00029.bin

 86%|████████████████████████████████████████████████████████████████████████████████████▏             | 415/483 [02:25<00:13,  5.16it/s]
                                                                                                                                         
[2024-01-30 23:25:29] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00025-of-00029.bin

 86%|████████████████████████████████████████████████████████████████████████████████████▏             | 415/483 [02:25<00:13,  5.16it/s]
                                                                                                                                         
[2024-01-30 23:25:31] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.67.input_layernorm.weight", shape: (8192,), dtype: float32

 86%|████████████████████████████████████████████████████████████████████████████████████▏             | 415/483 [02:26<00:13,  5.16it/s]
 86%|████████████████████████████████████████████████████████████████████████████████████▌             | 417/483 [02:26<00:26,  2.45it/s]
                                                                                                                                         
[2024-01-30 23:25:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.67.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 86%|████████████████████████████████████████████████████████████████████████████████████▌             | 417/483 [02:26<00:26,  2.45it/s]
                                                                                                                                         
[2024-01-30 23:25:31] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.67.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 86%|████████████████████████████████████████████████████████████████████████████████████▌             | 417/483 [02:26<00:26,  2.45it/s]
 87%|████████████████████████████████████████████████████████████████████████████████████▊             | 418/483 [02:26<00:23,  2.71it/s]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.67.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 87%|████████████████████████████████████████████████████████████████████████████████████▊             | 418/483 [02:31<00:23,  2.71it/s]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.67.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 87%|████████████████████████████████████████████████████████████████████████████████████▊             | 418/483 [02:31<00:23,  2.71it/s]
 87%|█████████████████████████████████████████████████████████████████████████████████████             | 419/483 [02:31<01:23,  1.31s/it]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.67.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 87%|█████████████████████████████████████████████████████████████████████████████████████             | 419/483 [02:31<01:23,  1.31s/it]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.67.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 87%|█████████████████████████████████████████████████████████████████████████████████████             | 419/483 [02:31<01:23,  1.31s/it]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.67.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 87%|█████████████████████████████████████████████████████████████████████████████████████             | 419/483 [02:31<01:23,  1.31s/it]
 87%|█████████████████████████████████████████████████████████████████████████████████████▍            | 421/483 [02:31<00:52,  1.19it/s]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.67.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 87%|█████████████████████████████████████████████████████████████████████████████████████▍            | 421/483 [02:31<00:52,  1.19it/s]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.67.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 87%|█████████████████████████████████████████████████████████████████████████████████████▍            | 421/483 [02:31<00:52,  1.19it/s]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.68.input_layernorm.weight", shape: (8192,), dtype: float32

 87%|█████████████████████████████████████████████████████████████████████████████████████▍            | 421/483 [02:31<00:52,  1.19it/s]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.68.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 87%|█████████████████████████████████████████████████████████████████████████████████████▍            | 421/483 [02:32<00:52,  1.19it/s]
                                                                                                                                         
[2024-01-30 23:25:36] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.68.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 87%|█████████████████████████████████████████████████████████████████████████████████████▍            | 421/483 [02:32<00:52,  1.19it/s]
 88%|██████████████████████████████████████████████████████████████████████████████████████            | 424/483 [02:32<00:30,  1.95it/s]
                                                                                                                                         
[2024-01-30 23:25:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.68.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 88%|██████████████████████████████████████████████████████████████████████████████████████            | 424/483 [02:32<00:30,  1.95it/s]
                                                                                                                                         
[2024-01-30 23:25:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.68.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 88%|██████████████████████████████████████████████████████████████████████████████████████            | 424/483 [02:33<00:30,  1.95it/s]
 88%|██████████████████████████████████████████████████████████████████████████████████████▏           | 425/483 [02:33<00:33,  1.74it/s]
                                                                                                                                         
[2024-01-30 23:25:37] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.68.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 88%|██████████████████████████████████████████████████████████████████████████████████████▏           | 425/483 [02:33<00:33,  1.74it/s]
                                                                                                                                         
[2024-01-30 23:25:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.68.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 88%|██████████████████████████████████████████████████████████████████████████████████████▏           | 425/483 [02:33<00:33,  1.74it/s]
                                                                                                                                         
[2024-01-30 23:25:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.68.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 88%|██████████████████████████████████████████████████████████████████████████████████████▏           | 425/483 [02:33<00:33,  1.74it/s]
 88%|██████████████████████████████████████████████████████████████████████████████████████▋           | 427/483 [02:33<00:22,  2.46it/s]
                                                                                                                                         
[2024-01-30 23:25:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.68.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 88%|██████████████████████████████████████████████████████████████████████████████████████▋           | 427/483 [02:33<00:22,  2.46it/s]
                                                                                                                                         
[2024-01-30 23:25:37] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.68.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 88%|██████████████████████████████████████████████████████████████████████████████████████▋           | 427/483 [02:33<00:22,  2.46it/s]
                                                                                                                                         
[2024-01-30 23:25:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.69.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 88%|██████████████████████████████████████████████████████████████████████████████████████▋           | 427/483 [02:34<00:22,  2.46it/s]
                                                                                                                                         
[2024-01-30 23:25:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.69.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 88%|██████████████████████████████████████████████████████████████████████████████████████▋           | 427/483 [02:34<00:22,  2.46it/s]
 89%|███████████████████████████████████████████████████████████████████████████████████████           | 429/483 [02:34<00:25,  2.09it/s]
                                                                                                                                         
[2024-01-30 23:25:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.69.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 89%|███████████████████████████████████████████████████████████████████████████████████████           | 429/483 [02:34<00:25,  2.09it/s]
                                                                                                                                         
[2024-01-30 23:25:38] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.69.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 89%|███████████████████████████████████████████████████████████████████████████████████████           | 429/483 [02:34<00:25,  2.09it/s]
 89%|███████████████████████████████████████████████████████████████████████████████████████▏          | 430/483 [02:34<00:22,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:25:39] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.69.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 89%|███████████████████████████████████████████████████████████████████████████████████████▏          | 430/483 [02:34<00:22,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:25:39] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.69.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 89%|███████████████████████████████████████████████████████████████████████████████████████▏          | 430/483 [02:34<00:22,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:25:39] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00025-of-00029.bin

 89%|███████████████████████████████████████████████████████████████████████████████████████▏          | 430/483 [02:34<00:22,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:25:39] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00026-of-00029.bin

 89%|███████████████████████████████████████████████████████████████████████████████████████▏          | 430/483 [02:34<00:22,  2.40it/s]
                                                                                                                                         
[2024-01-30 23:25:40] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.69.input_layernorm.weight", shape: (8192,), dtype: float32

 89%|███████████████████████████████████████████████████████████████████████████████████████▏          | 430/483 [02:36<00:22,  2.40it/s]
 89%|███████████████████████████████████████████████████████████████████████████████████████▋          | 432/483 [02:36<00:29,  1.75it/s]
                                                                                                                                         
[2024-01-30 23:25:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.69.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 89%|███████████████████████████████████████████████████████████████████████████████████████▋          | 432/483 [02:36<00:29,  1.75it/s]
                                                                                                                                         
[2024-01-30 23:25:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.69.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 89%|███████████████████████████████████████████████████████████████████████████████████████▋          | 432/483 [02:36<00:29,  1.75it/s]
 90%|███████████████████████████████████████████████████████████████████████████████████████▊          | 433/483 [02:36<00:26,  1.90it/s]
                                                                                                                                         
[2024-01-30 23:25:41] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.69.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 90%|███████████████████████████████████████████████████████████████████████████████████████▊          | 433/483 [02:36<00:26,  1.90it/s]
                                                                                                                                         
[2024-01-30 23:25:41] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.70.input_layernorm.weight", shape: (8192,), dtype: float32

 90%|███████████████████████████████████████████████████████████████████████████████████████▊          | 433/483 [02:36<00:26,  1.90it/s]
                                                                                                                                         
[2024-01-30 23:25:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.70.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 90%|███████████████████████████████████████████████████████████████████████████████████████▊          | 433/483 [02:36<00:26,  1.90it/s]
                                                                                                                                         
[2024-01-30 23:25:41] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.70.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 90%|███████████████████████████████████████████████████████████████████████████████████████▊          | 433/483 [02:36<00:26,  1.90it/s]
 90%|████████████████████████████████████████████████████████████████████████████████████████▍         | 436/483 [02:36<00:15,  3.10it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.70.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 90%|████████████████████████████████████████████████████████████████████████████████████████▍         | 436/483 [02:37<00:15,  3.10it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.70.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 90%|████████████████████████████████████████████████████████████████████████████████████████▍         | 436/483 [02:37<00:15,  3.10it/s]
 90%|████████████████████████████████████████████████████████████████████████████████████████▋         | 437/483 [02:37<00:19,  2.34it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.70.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 90%|████████████████████████████████████████████████████████████████████████████████████████▋         | 437/483 [02:37<00:19,  2.34it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.70.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 90%|████████████████████████████████████████████████████████████████████████████████████████▋         | 437/483 [02:37<00:19,  2.34it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.70.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 90%|████████████████████████████████████████████████████████████████████████████████████████▋         | 437/483 [02:37<00:19,  2.34it/s]
 91%|█████████████████████████████████████████████████████████████████████████████████████████         | 439/483 [02:37<00:13,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.70.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 91%|█████████████████████████████████████████████████████████████████████████████████████████         | 439/483 [02:37<00:13,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.70.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 91%|█████████████████████████████████████████████████████████████████████████████████████████         | 439/483 [02:37<00:13,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.71.input_layernorm.weight", shape: (8192,), dtype: float32

 91%|█████████████████████████████████████████████████████████████████████████████████████████         | 439/483 [02:38<00:13,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.71.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 91%|█████████████████████████████████████████████████████████████████████████████████████████         | 439/483 [02:38<00:13,  3.26it/s]
                                                                                                                                         
[2024-01-30 23:25:42] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.71.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 91%|█████████████████████████████████████████████████████████████████████████████████████████         | 439/483 [02:38<00:13,  3.26it/s]
 92%|█████████████████████████████████████████████████████████████████████████████████████████▋        | 442/483 [02:38<00:09,  4.47it/s]
                                                                                                                                         
[2024-01-30 23:25:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.71.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 92%|█████████████████████████████████████████████████████████████████████████████████████████▋        | 442/483 [02:39<00:09,  4.47it/s]
                                                                                                                                         
[2024-01-30 23:25:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.71.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 92%|█████████████████████████████████████████████████████████████████████████████████████████▋        | 442/483 [02:39<00:09,  4.47it/s]
 92%|█████████████████████████████████████████████████████████████████████████████████████████▉        | 443/483 [02:39<00:13,  3.04it/s]
                                                                                                                                         
[2024-01-30 23:25:43] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.71.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 92%|█████████████████████████████████████████████████████████████████████████████████████████▉        | 443/483 [02:39<00:13,  3.04it/s]
                                                                                                                                         
[2024-01-30 23:25:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.71.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 92%|█████████████████████████████████████████████████████████████████████████████████████████▉        | 443/483 [02:39<00:13,  3.04it/s]
                                                                                                                                         
[2024-01-30 23:25:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.71.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 92%|█████████████████████████████████████████████████████████████████████████████████████████▉        | 443/483 [02:39<00:13,  3.04it/s]
 92%|██████████████████████████████████████████████████████████████████████████████████████████▎       | 445/483 [02:39<00:09,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:25:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.71.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 92%|██████████████████████████████████████████████████████████████████████████████████████████▎       | 445/483 [02:39<00:09,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:25:43] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.71.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 92%|██████████████████████████████████████████████████████████████████████████████████████████▎       | 445/483 [02:39<00:09,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:25:43] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00027-of-00029.bin

 92%|██████████████████████████████████████████████████████████████████████████████████████████▎       | 445/483 [02:39<00:09,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:25:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.72.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 92%|██████████████████████████████████████████████████████████████████████████████████████████▎       | 445/483 [02:48<00:09,  4.09it/s]
                                                                                                                                         
[2024-01-30 23:25:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.72.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 92%|██████████████████████████████████████████████████████████████████████████████████████████▎       | 445/483 [02:48<00:09,  4.09it/s]
 93%|██████████████████████████████████████████████████████████████████████████████████████████▋       | 447/483 [02:48<00:57,  1.59s/it]
                                                                                                                                         
[2024-01-30 23:25:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.72.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 93%|██████████████████████████████████████████████████████████████████████████████████████████▋       | 447/483 [02:48<00:57,  1.59s/it]
                                                                                                                                         
[2024-01-30 23:25:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.72.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 93%|██████████████████████████████████████████████████████████████████████████████████████████▋       | 447/483 [02:48<00:57,  1.59s/it]
 93%|██████████████████████████████████████████████████████████████████████████████████████████▉       | 448/483 [02:48<00:46,  1.33s/it]
                                                                                                                                         
[2024-01-30 23:25:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.72.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 93%|██████████████████████████████████████████████████████████████████████████████████████████▉       | 448/483 [02:48<00:46,  1.33s/it]
                                                                                                                                         
[2024-01-30 23:25:52] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.72.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 93%|██████████████████████████████████████████████████████████████████████████████████████████▉       | 448/483 [02:48<00:46,  1.33s/it]
                                                                                                                                         
[2024-01-30 23:25:52] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.72.input_layernorm.weight", shape: (8192,), dtype: float32

 93%|██████████████████████████████████████████████████████████████████████████████████████████▉       | 448/483 [02:48<00:46,  1.33s/it]
                                                                                                                                         
[2024-01-30 23:25:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.72.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 93%|██████████████████████████████████████████████████████████████████████████████████████████▉       | 448/483 [02:49<00:46,  1.33s/it]
                                                                                                                                         
[2024-01-30 23:25:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.72.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 93%|██████████████████████████████████████████████████████████████████████████████████████████▉       | 448/483 [02:49<00:46,  1.33s/it]
 93%|███████████████████████████████████████████████████████████████████████████████████████████▌      | 451/483 [02:49<00:26,  1.20it/s]
                                                                                                                                         
[2024-01-30 23:25:53] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.72.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 93%|███████████████████████████████████████████████████████████████████████████████████████████▌      | 451/483 [02:49<00:26,  1.20it/s]
                                                                                                                                         
[2024-01-30 23:25:53] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.73.input_layernorm.weight", shape: (8192,), dtype: float32

 93%|███████████████████████████████████████████████████████████████████████████████████████████▌      | 451/483 [02:49<00:26,  1.20it/s]
                                                                                                                                         
[2024-01-30 23:25:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.73.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 93%|███████████████████████████████████████████████████████████████████████████████████████████▌      | 451/483 [02:49<00:26,  1.20it/s]
                                                                                                                                         
[2024-01-30 23:25:53] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.73.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 93%|███████████████████████████████████████████████████████████████████████████████████████████▌      | 451/483 [02:49<00:26,  1.20it/s]
 94%|████████████████████████████████████████████████████████████████████████████████████████████      | 454/483 [02:49<00:16,  1.72it/s]
                                                                                                                                         
[2024-01-30 23:25:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.73.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 94%|████████████████████████████████████████████████████████████████████████████████████████████      | 454/483 [02:50<00:16,  1.72it/s]
                                                                                                                                         
[2024-01-30 23:25:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.73.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 94%|████████████████████████████████████████████████████████████████████████████████████████████      | 454/483 [02:50<00:16,  1.72it/s]
 94%|████████████████████████████████████████████████████████████████████████████████████████████▎     | 455/483 [02:50<00:19,  1.41it/s]
                                                                                                                                         
[2024-01-30 23:25:55] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.73.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 94%|████████████████████████████████████████████████████████████████████████████████████████████▎     | 455/483 [02:51<00:19,  1.41it/s]
                                                                                                                                         
[2024-01-30 23:25:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.73.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 94%|████████████████████████████████████████████████████████████████████████████████████████████▎     | 455/483 [02:51<00:19,  1.41it/s]
                                                                                                                                         
[2024-01-30 23:25:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.73.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 94%|████████████████████████████████████████████████████████████████████████████████████████████▎     | 455/483 [02:51<00:19,  1.41it/s]
 95%|████████████████████████████████████████████████████████████████████████████████████████████▋     | 457/483 [02:51<00:13,  1.90it/s]
                                                                                                                                         
[2024-01-30 23:25:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.73.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 95%|████████████████████████████████████████████████████████████████████████████████████████████▋     | 457/483 [02:51<00:13,  1.90it/s]
                                                                                                                                         
[2024-01-30 23:25:55] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.73.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 95%|████████████████████████████████████████████████████████████████████████████████████████████▋     | 457/483 [02:51<00:13,  1.90it/s]
 95%|████████████████████████████████████████████████████████████████████████████████████████████▉     | 458/483 [02:51<00:11,  2.19it/s]
                                                                                                                                         
[2024-01-30 23:25:55] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.74.input_layernorm.weight", shape: (8192,), dtype: float32

 95%|████████████████████████████████████████████████████████████████████████████████████████████▉     | 458/483 [02:51<00:11,  2.19it/s]
                                                                                                                                         
[2024-01-30 23:25:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.74.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 95%|████████████████████████████████████████████████████████████████████████████████████████████▉     | 458/483 [02:51<00:11,  2.19it/s]
                                                                                                                                         
[2024-01-30 23:25:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.74.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 95%|████████████████████████████████████████████████████████████████████████████████████████████▉     | 458/483 [02:51<00:11,  2.19it/s]
 95%|█████████████████████████████████████████████████████████████████████████████████████████████▎    | 460/483 [02:51<00:08,  2.84it/s]
                                                                                                                                         
[2024-01-30 23:25:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.74.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 95%|█████████████████████████████████████████████████████████████████████████████████████████████▎    | 460/483 [02:52<00:08,  2.84it/s]
                                                                                                                                         
[2024-01-30 23:25:56] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.74.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 95%|█████████████████████████████████████████████████████████████████████████████████████████████▎    | 460/483 [02:52<00:08,  2.84it/s]
 95%|█████████████████████████████████████████████████████████████████████████████████████████████▌    | 461/483 [02:52<00:10,  2.15it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.74.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 95%|█████████████████████████████████████████████████████████████████████████████████████████████▌    | 461/483 [02:52<00:10,  2.15it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.74.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 95%|█████████████████████████████████████████████████████████████████████████████████████████████▌    | 461/483 [02:52<00:10,  2.15it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.74.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 95%|█████████████████████████████████████████████████████████████████████████████████████████████▌    | 461/483 [02:52<00:10,  2.15it/s]
 96%|█████████████████████████████████████████████████████████████████████████████████████████████▉    | 463/483 [02:52<00:06,  2.96it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.74.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 96%|█████████████████████████████████████████████████████████████████████████████████████████████▉    | 463/483 [02:52<00:06,  2.96it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.74.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 96%|█████████████████████████████████████████████████████████████████████████████████████████████▉    | 463/483 [02:52<00:06,  2.96it/s]
 96%|██████████████████████████████████████████████████████████████████████████████████████████████▏   | 464/483 [02:52<00:05,  3.35it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.75.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 96%|██████████████████████████████████████████████████████████████████████████████████████████████▏   | 464/483 [02:53<00:05,  3.35it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.75.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 96%|██████████████████████████████████████████████████████████████████████████████████████████████▏   | 464/483 [02:53<00:05,  3.35it/s]
 96%|██████████████████████████████████████████████████████████████████████████████████████████████▎   | 465/483 [02:53<00:04,  3.88it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.75.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 96%|██████████████████████████████████████████████████████████████████████████████████████████████▎   | 465/483 [02:53<00:04,  3.88it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.75.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 96%|██████████████████████████████████████████████████████████████████████████████████████████████▎   | 465/483 [02:53<00:04,  3.88it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00027-of-00029.bin

 96%|██████████████████████████████████████████████████████████████████████████████████████████████▎   | 465/483 [02:53<00:04,  3.88it/s]
                                                                                                                                         
[2024-01-30 23:25:57] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00026-of-00029.bin

 96%|██████████████████████████████████████████████████████████████████████████████████████████████▎   | 465/483 [02:53<00:04,  3.88it/s]
                                                                                                                                         
[2024-01-30 23:25:58] INFO huggingface_loader.py:169: Loading HF parameters from: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00028-of-00029.bin

 96%|██████████████████████████████████████████████████████████████████████████████████████████████▎   | 465/483 [02:53<00:04,  3.88it/s]
                                                                                                                                         
[2024-01-30 23:25:59] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.75.input_layernorm.weight", shape: (8192,), dtype: float32

 96%|██████████████████████████████████████████████████████████████████████████████████████████████▎   | 465/483 [02:55<00:04,  3.88it/s]
 97%|██████████████████████████████████████████████████████████████████████████████████████████████▊   | 467/483 [02:55<00:09,  1.74it/s]
                                                                                                                                         
[2024-01-30 23:25:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.75.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 97%|██████████████████████████████████████████████████████████████████████████████████████████████▊   | 467/483 [02:55<00:09,  1.74it/s]
                                                                                                                                         
[2024-01-30 23:25:59] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.75.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 97%|██████████████████████████████████████████████████████████████████████████████████████████████▊   | 467/483 [02:55<00:09,  1.74it/s]
 97%|██████████████████████████████████████████████████████████████████████████████████████████████▉   | 468/483 [02:55<00:07,  2.02it/s]
                                                                                                                                         
[2024-01-30 23:26:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.75.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 97%|██████████████████████████████████████████████████████████████████████████████████████████████▉   | 468/483 [02:56<00:07,  2.02it/s]
                                                                                                                                         
[2024-01-30 23:26:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.75.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 97%|██████████████████████████████████████████████████████████████████████████████████████████████▉   | 468/483 [02:56<00:07,  2.02it/s]
 97%|███████████████████████████████████████████████████████████████████████████████████████████████▏  | 469/483 [02:56<00:07,  1.84it/s]
                                                                                                                                         
[2024-01-30 23:26:00] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.75.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 97%|███████████████████████████████████████████████████████████████████████████████████████████████▏  | 469/483 [02:56<00:07,  1.84it/s]
                                                                                                                                         
[2024-01-30 23:26:00] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.76.input_layernorm.weight", shape: (8192,), dtype: float32

 97%|███████████████████████████████████████████████████████████████████████████████████████████████▏  | 469/483 [02:56<00:07,  1.84it/s]
                                                                                                                                         
[2024-01-30 23:26:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.76.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 97%|███████████████████████████████████████████████████████████████████████████████████████████████▏  | 469/483 [02:56<00:07,  1.84it/s]
                                                                                                                                         
[2024-01-30 23:26:00] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.76.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 97%|███████████████████████████████████████████████████████████████████████████████████████████████▏  | 469/483 [02:56<00:07,  1.84it/s]
 98%|███████████████████████████████████████████████████████████████████████████████████████████████▊  | 472/483 [02:56<00:03,  3.17it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.76.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 98%|███████████████████████████████████████████████████████████████████████████████████████████████▊  | 472/483 [02:57<00:03,  3.17it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.76.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 98%|███████████████████████████████████████████████████████████████████████████████████████████████▊  | 472/483 [02:57<00:03,  3.17it/s]
 98%|███████████████████████████████████████████████████████████████████████████████████████████████▉  | 473/483 [02:57<00:03,  2.55it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.76.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 98%|███████████████████████████████████████████████████████████████████████████████████████████████▉  | 473/483 [02:57<00:03,  2.55it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.76.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 98%|███████████████████████████████████████████████████████████████████████████████████████████████▉  | 473/483 [02:57<00:03,  2.55it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.76.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 98%|███████████████████████████████████████████████████████████████████████████████████████████████▉  | 473/483 [02:57<00:03,  2.55it/s]
 98%|████████████████████████████████████████████████████████████████████████████████████████████████▍ | 475/483 [02:57<00:02,  3.61it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.76.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

 98%|████████████████████████████████████████████████████████████████████████████████████████████████▍ | 475/483 [02:57<00:02,  3.61it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.76.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

 98%|████████████████████████████████████████████████████████████████████████████████████████████████▍ | 475/483 [02:57<00:02,  3.61it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.77.input_layernorm.weight", shape: (8192,), dtype: float32

 98%|████████████████████████████████████████████████████████████████████████████████████████████████▍ | 475/483 [02:57<00:02,  3.61it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.77.mlp.down_proj.q_weight", shape: (8192, 3584), dtype: uint32

 98%|████████████████████████████████████████████████████████████████████████████████████████████████▍ | 475/483 [02:57<00:02,  3.61it/s]
                                                                                                                                         
[2024-01-30 23:26:01] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.77.mlp.down_proj.q_scale", shape: (8192, 896), dtype: float32

 98%|████████████████████████████████████████████████████████████████████████████████████████████████▍ | 475/483 [02:57<00:02,  3.61it/s]
 99%|████████████████████████████████████████████████████████████████████████████████████████████████▉ | 478/483 [02:57<00:01,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:26:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.77.mlp.gate_up_proj.q_weight", shape: (57344, 1024), dtype: uint32

 99%|████████████████████████████████████████████████████████████████████████████████████████████████▉ | 478/483 [02:58<00:01,  4.95it/s]
                                                                                                                                         
[2024-01-30 23:26:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.77.mlp.gate_up_proj.q_scale", shape: (57344, 256), dtype: float32

 99%|████████████████████████████████████████████████████████████████████████████████████████████████▉ | 478/483 [02:58<00:01,  4.95it/s]
 99%|█████████████████████████████████████████████████████████████████████████████████████████████████▏| 479/483 [02:58<00:01,  3.41it/s]
                                                                                                                                         
[2024-01-30 23:26:02] INFO huggingface_loader.py:129: [Not quantized] Parameter: "model.layers.77.post_attention_layernorm.weight", shape: (8192,), dtype: float32

 99%|█████████████████████████████████████████████████████████████████████████████████████████████████▏| 479/483 [02:58<00:01,  3.41it/s]
                                                                                                                                         
[2024-01-30 23:26:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.77.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

 99%|█████████████████████████████████████████████████████████████████████████████████████████████████▏| 479/483 [02:58<00:01,  3.41it/s]
                                                                                                                                         
[2024-01-30 23:26:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.77.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

 99%|█████████████████████████████████████████████████████████████████████████████████████████████████▏| 479/483 [02:58<00:01,  3.41it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████▌| 481/483 [02:58<00:00,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:26:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.77.self_attn.o_proj.q_weight", shape: (8192, 1024), dtype: uint32

100%|█████████████████████████████████████████████████████████████████████████████████████████████████▌| 481/483 [02:58<00:00,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:26:02] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.77.self_attn.o_proj.q_scale", shape: (8192, 256), dtype: float32

100%|█████████████████████████████████████████████████████████████████████████████████████████████████▌| 481/483 [02:58<00:00,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:26:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.78.self_attn.qkv_proj.q_weight", shape: (10240, 1024), dtype: uint32

100%|█████████████████████████████████████████████████████████████████████████████████████████████████▌| 481/483 [02:58<00:00,  4.56it/s]
                                                                                                                                         
[2024-01-30 23:26:03] INFO huggingface_loader.py:121: [Quantized] Parameter: "model.layers.78.self_attn.qkv_proj.q_scale", shape: (10240, 256), dtype: float32

100%|█████████████████████████████████████████████████████████████████████████████████████████████████▌| 481/483 [02:58<00:00,  4.56it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 483/483 [02:58<00:00,  5.61it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 483/483 [02:58<00:00,  2.70it/s]
[2024-01-30 23:26:03] INFO huggingface_loader.py:181: Unloading HF weight file: /Users/Shared/models/CodeLlama-70b-Python-hf/pytorch_model-00028-of-00029.bin
[2024-01-30 23:26:03] INFO stats.py:76: Time usage: HF loading: 60.238 sec; Pre-quantization mapping: 67.483 sec; Quantization: 5.030 sec
[2024-01-30 23:26:03] INFO stats.py:90: RAM usage: Peak RAM: 17.477 GB. Total bytes loaded from disk: 265.647 GB
[2024-01-30 23:26:03] INFO convert_weight.py:121: Parameter size after quantization: 40.154 GB
[2024-01-30 23:26:03] INFO convert_weight.py:126: Total parameters: 71,308,992,512
[2024-01-30 23:26:03] INFO convert_weight.py:127: Bits per parameter: 4.837
Start storing to cache /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq


[0001/0805] saving lm_head.q_weight
                                   
[0002/0805] saving lm_head.q_scale
                                   
[0003/0805] saving model.layers.78.input_layernorm.weight
                                                         
[0004/0805] saving model.layers.78.mlp.down_proj.q_weight
                                                         
[0005/0805] saving model.layers.78.mlp.down_proj.q_scale
                                                         
[0006/0805] saving model.layers.78.mlp.gate_up_proj.q_weight
                                                            
[0007/0805] saving model.layers.78.mlp.gate_up_proj.q_scale
                                                            
[0008/0805] saving model.layers.78.post_attention_layernorm.weight
                                                                  
[0009/0805] saving model.layers.78.self_attn.o_proj.q_weight
                                                                  
[0010/0805] saving model.layers.78.self_attn.o_proj.q_scale
                                                                  
[0011/0805] saving model.layers.79.input_layernorm.weight
                                                                  
[0012/0805] saving model.layers.79.mlp.down_proj.q_weight
                                                                  
[0013/0805] saving model.layers.79.mlp.down_proj.q_scale
                                                                  
[0014/0805] saving model.layers.79.mlp.gate_up_proj.q_weight
                                                                  
[0015/0805] saving model.layers.79.mlp.gate_up_proj.q_scale
                                                                  
[0016/0805] saving model.layers.79.post_attention_layernorm.weight
                                                                  
[0017/0805] saving model.layers.79.self_attn.qkv_proj.q_weight
                                                                  
[0018/0805] saving model.layers.79.self_attn.qkv_proj.q_scale
                                                                  
[0019/0805] saving model.layers.79.self_attn.o_proj.q_weight
                                                                  
[0020/0805] saving model.layers.79.self_attn.o_proj.q_scale
                                                                  
[0021/0805] saving model.norm.weight
                                                                  
[0022/0805] saving model.embed_tokens.q_weight
                                                                  
[0023/0805] saving model.embed_tokens.q_scale
                                                                  
[0024/0805] saving model.layers.0.input_layernorm.weight
                                                                  
[0025/0805] saving model.layers.0.mlp.down_proj.q_weight
                                                                  
[0026/0805] saving model.layers.0.mlp.down_proj.q_scale
                                                                  
[0027/0805] saving model.layers.0.mlp.gate_up_proj.q_weight
                                                                  
[0028/0805] saving model.layers.0.mlp.gate_up_proj.q_scale
                                                                  
[0029/0805] saving model.layers.0.post_attention_layernorm.weight
                                                                  
[0030/0805] saving model.layers.0.self_attn.qkv_proj.q_weight
                                                                  
[0031/0805] saving model.layers.0.self_attn.qkv_proj.q_scale
                                                                  
[0032/0805] saving model.layers.0.self_attn.o_proj.q_weight
                                                                  
[0033/0805] saving model.layers.0.self_attn.o_proj.q_scale
                                                                  
[0034/0805] saving model.layers.1.input_layernorm.weight
                                                                  
[0035/0805] saving model.layers.1.mlp.down_proj.q_weight
                                                                  
[0036/0805] saving model.layers.1.mlp.down_proj.q_scale
                                                                  
[0037/0805] saving model.layers.1.mlp.gate_up_proj.q_weight
                                                                  
[0038/0805] saving model.layers.1.mlp.gate_up_proj.q_scale
                                                                  
[0039/0805] saving model.layers.1.post_attention_layernorm.weight
                                                                  
[0040/0805] saving model.layers.1.self_attn.qkv_proj.q_weight
                                                                  
[0041/0805] saving model.layers.1.self_attn.qkv_proj.q_scale
                                                                  
[0042/0805] saving model.layers.1.self_attn.o_proj.q_weight
                                                                  
[0043/0805] saving model.layers.1.self_attn.o_proj.q_scale
                                                                  
[0044/0805] saving model.layers.2.mlp.gate_up_proj.q_weight
                                                                  
[0045/0805] saving model.layers.2.mlp.gate_up_proj.q_scale
                                                                  
[0046/0805] saving model.layers.2.self_attn.qkv_proj.q_weight
                                                                  
[0047/0805] saving model.layers.2.self_attn.qkv_proj.q_scale
                                                                  
[0048/0805] saving model.layers.2.self_attn.o_proj.q_weight
                                                                  
[0049/0805] saving model.layers.2.self_attn.o_proj.q_scale
                                                                  
[0050/0805] saving model.layers.10.input_layernorm.weight
                                                                  
[0051/0805] saving model.layers.10.mlp.down_proj.q_weight
                                                                  
[0052/0805] saving model.layers.10.mlp.down_proj.q_scale
                                                                  
[0053/0805] saving model.layers.10.mlp.gate_up_proj.q_weight
                                                                  
[0054/0805] saving model.layers.10.mlp.gate_up_proj.q_scale
                                                                  
[0055/0805] saving model.layers.10.post_attention_layernorm.weight
                                                                  
[0056/0805] saving model.layers.10.self_attn.qkv_proj.q_weight
                                                                  
[0057/0805] saving model.layers.10.self_attn.qkv_proj.q_scale
                                                                  
[0058/0805] saving model.layers.10.self_attn.o_proj.q_weight
                                                                  
[0059/0805] saving model.layers.10.self_attn.o_proj.q_scale
                                                                  
[0060/0805] saving model.layers.8.input_layernorm.weight
                                                                  
[0061/0805] saving model.layers.8.mlp.down_proj.q_weight
                                                                  
[0062/0805] saving model.layers.8.mlp.down_proj.q_scale
                                                                  
[0063/0805] saving model.layers.8.mlp.gate_up_proj.q_weight
                                                                  
[0064/0805] saving model.layers.8.mlp.gate_up_proj.q_scale
                                                                  
[0065/0805] saving model.layers.8.post_attention_layernorm.weight
                                                                  
[0066/0805] saving model.layers.8.self_attn.o_proj.q_weight
                                                                  
[0067/0805] saving model.layers.8.self_attn.o_proj.q_scale
                                                                  
[0068/0805] saving model.layers.9.input_layernorm.weight
                                                                  
[0069/0805] saving model.layers.9.mlp.down_proj.q_weight
                                                                  
[0070/0805] saving model.layers.9.mlp.down_proj.q_scale
                                                                  
[0071/0805] saving model.layers.9.mlp.gate_up_proj.q_weight
                                                                  
[0072/0805] saving model.layers.9.mlp.gate_up_proj.q_scale
                                                                  
[0073/0805] saving model.layers.9.post_attention_layernorm.weight
                                                                  
[0074/0805] saving model.layers.9.self_attn.qkv_proj.q_weight
                                                                  
[0075/0805] saving model.layers.9.self_attn.qkv_proj.q_scale
                                                                  
[0076/0805] saving model.layers.9.self_attn.o_proj.q_weight
                                                                  
[0077/0805] saving model.layers.9.self_attn.o_proj.q_scale
                                                                  
[0078/0805] saving model.layers.11.input_layernorm.weight
                                                                  
[0079/0805] saving model.layers.11.mlp.down_proj.q_weight
                                                                  
[0080/0805] saving model.layers.11.mlp.down_proj.q_scale
                                                                  
[0081/0805] saving model.layers.11.mlp.gate_up_proj.q_weight
                                                                  
[0082/0805] saving model.layers.11.mlp.gate_up_proj.q_scale
                                                                  
[0083/0805] saving model.layers.11.post_attention_layernorm.weight
                                                                  
[0084/0805] saving model.layers.11.self_attn.qkv_proj.q_weight
                                                                  
[0085/0805] saving model.layers.11.self_attn.qkv_proj.q_scale
                                                                  
[0086/0805] saving model.layers.11.self_attn.o_proj.q_weight
                                                                  
[0087/0805] saving model.layers.11.self_attn.o_proj.q_scale
                                                                  
[0088/0805] saving model.layers.12.input_layernorm.weight
                                                                  
[0089/0805] saving model.layers.12.mlp.down_proj.q_weight
                                                                  
[0090/0805] saving model.layers.12.mlp.down_proj.q_scale
                                                                  
[0091/0805] saving model.layers.12.mlp.gate_up_proj.q_weight
                                                                  
[0092/0805] saving model.layers.12.mlp.gate_up_proj.q_scale
                                                                  
[0093/0805] saving model.layers.12.post_attention_layernorm.weight
                                                                  
[0094/0805] saving model.layers.12.self_attn.qkv_proj.q_weight
                                                                  
[0095/0805] saving model.layers.12.self_attn.qkv_proj.q_scale
                                                                  
[0096/0805] saving model.layers.12.self_attn.o_proj.q_weight
                                                                  
[0097/0805] saving model.layers.12.self_attn.o_proj.q_scale
                                                                  
[0098/0805] saving model.layers.13.mlp.gate_up_proj.q_weight
                                                                  
[0099/0805] saving model.layers.13.mlp.gate_up_proj.q_scale
                                                                  
[0100/0805] saving model.layers.13.self_attn.qkv_proj.q_weight
                                                                  
[0101/0805] saving model.layers.13.self_attn.qkv_proj.q_scale
                                                                  
[0102/0805] saving model.layers.13.self_attn.o_proj.q_weight
                                                                  
[0103/0805] saving model.layers.13.self_attn.o_proj.q_scale
                                                                  
[0104/0805] saving model.layers.13.input_layernorm.weight
                                                                  
[0105/0805] saving model.layers.13.mlp.down_proj.q_weight
                                                                  
[0106/0805] saving model.layers.13.mlp.down_proj.q_scale
                                                                  
[0107/0805] saving model.layers.13.post_attention_layernorm.weight
                                                                  
[0108/0805] saving model.layers.14.input_layernorm.weight
                                                                  
[0109/0805] saving model.layers.14.mlp.down_proj.q_weight
                                                                  
[0110/0805] saving model.layers.14.mlp.down_proj.q_scale
                                                                  
[0111/0805] saving model.layers.14.mlp.gate_up_proj.q_weight
                                                                  
[0112/0805] saving model.layers.14.mlp.gate_up_proj.q_scale
                                                                  
[0113/0805] saving model.layers.14.post_attention_layernorm.weight
                                                                  
[0114/0805] saving model.layers.14.self_attn.qkv_proj.q_weight
                                                                  
[0115/0805] saving model.layers.14.self_attn.qkv_proj.q_scale
                                                                  
[0116/0805] saving model.layers.14.self_attn.o_proj.q_weight
                                                                  
[0117/0805] saving model.layers.14.self_attn.o_proj.q_scale
                                                                  
[0118/0805] saving model.layers.15.input_layernorm.weight
                                                                  
[0119/0805] saving model.layers.15.mlp.down_proj.q_weight
                                                                  
[0120/0805] saving model.layers.15.mlp.down_proj.q_scale
                                                                  
[0121/0805] saving model.layers.15.mlp.gate_up_proj.q_weight
                                                                  
[0122/0805] saving model.layers.15.mlp.gate_up_proj.q_scale
                                                                  
[0123/0805] saving model.layers.15.post_attention_layernorm.weight
                                                                  
[0124/0805] saving model.layers.15.self_attn.qkv_proj.q_weight
                                                                  
[0125/0805] saving model.layers.15.self_attn.qkv_proj.q_scale
                                                                  
[0126/0805] saving model.layers.15.self_attn.o_proj.q_weight
                                                                  
[0127/0805] saving model.layers.15.self_attn.o_proj.q_scale
                                                                  
[0128/0805] saving model.layers.16.mlp.gate_up_proj.q_weight
                                                                  
[0129/0805] saving model.layers.16.mlp.gate_up_proj.q_scale
                                                                  
[0130/0805] saving model.layers.16.self_attn.qkv_proj.q_weight
                                                                  
[0131/0805] saving model.layers.16.self_attn.qkv_proj.q_scale
                                                                  
[0132/0805] saving model.layers.16.self_attn.o_proj.q_weight
                                                                  
[0133/0805] saving model.layers.16.self_attn.o_proj.q_scale
                                                                  
[0134/0805] saving model.layers.16.input_layernorm.weight
                                                                  
[0135/0805] saving model.layers.16.mlp.down_proj.q_weight
                                                                  
[0136/0805] saving model.layers.16.mlp.down_proj.q_scale
                                                                  
[0137/0805] saving model.layers.16.post_attention_layernorm.weight
                                                                  
[0138/0805] saving model.layers.17.input_layernorm.weight
                                                                  
[0139/0805] saving model.layers.17.mlp.down_proj.q_weight
                                                                  
[0140/0805] saving model.layers.17.mlp.down_proj.q_scale
                                                                  
[0141/0805] saving model.layers.17.mlp.gate_up_proj.q_weight
                                                                  
[0142/0805] saving model.layers.17.mlp.gate_up_proj.q_scale
                                                                  
[0143/0805] saving model.layers.17.post_attention_layernorm.weight
                                                                  
[0144/0805] saving model.layers.17.self_attn.qkv_proj.q_weight
                                                                  
[0145/0805] saving model.layers.17.self_attn.qkv_proj.q_scale
                                                                  
[0146/0805] saving model.layers.17.self_attn.o_proj.q_weight
                                                                  
[0147/0805] saving model.layers.17.self_attn.o_proj.q_scale
                                                                  
[0148/0805] saving model.layers.18.input_layernorm.weight
                                                                  
[0149/0805] saving model.layers.18.mlp.down_proj.q_weight
                                                                  
[0150/0805] saving model.layers.18.mlp.down_proj.q_scale
                                                                  
[0151/0805] saving model.layers.18.mlp.gate_up_proj.q_weight
                                                                  
[0152/0805] saving model.layers.18.mlp.gate_up_proj.q_scale
                                                                  
[0153/0805] saving model.layers.18.post_attention_layernorm.weight
                                                                  
[0154/0805] saving model.layers.18.self_attn.qkv_proj.q_weight
                                                                  
[0155/0805] saving model.layers.18.self_attn.qkv_proj.q_scale
                                                                  
[0156/0805] saving model.layers.18.self_attn.o_proj.q_weight
                                                                  
[0157/0805] saving model.layers.18.self_attn.o_proj.q_scale
                                                                  
[0158/0805] saving model.layers.19.self_attn.qkv_proj.q_weight
                                                                  
[0159/0805] saving model.layers.19.self_attn.qkv_proj.q_scale
                                                                  
[0160/0805] saving model.layers.19.self_attn.o_proj.q_weight
                                                                  
[0161/0805] saving model.layers.19.self_attn.o_proj.q_scale
                                                                  
[0162/0805] saving model.layers.19.input_layernorm.weight
                                                                  
[0163/0805] saving model.layers.19.mlp.down_proj.q_weight
                                                                  
[0164/0805] saving model.layers.19.mlp.down_proj.q_scale
                                                                  
[0165/0805] saving model.layers.19.mlp.gate_up_proj.q_weight
                                                                  
[0166/0805] saving model.layers.19.mlp.gate_up_proj.q_scale
                                                                  
[0167/0805] saving model.layers.19.post_attention_layernorm.weight
                                                                  
[0168/0805] saving model.layers.20.input_layernorm.weight
                                                                  
[0169/0805] saving model.layers.20.mlp.down_proj.q_weight
                                                                  
[0170/0805] saving model.layers.20.mlp.down_proj.q_scale
                                                                  
[0171/0805] saving model.layers.20.mlp.gate_up_proj.q_weight
                                                                  
[0172/0805] saving model.layers.20.mlp.gate_up_proj.q_scale
                                                                  
[0173/0805] saving model.layers.20.post_attention_layernorm.weight
                                                                  
[0174/0805] saving model.layers.20.self_attn.qkv_proj.q_weight
                                                                  
[0175/0805] saving model.layers.20.self_attn.qkv_proj.q_scale
                                                                  
[0176/0805] saving model.layers.20.self_attn.o_proj.q_weight
                                                                  
[0177/0805] saving model.layers.20.self_attn.o_proj.q_scale
                                                                  
[0178/0805] saving model.layers.21.input_layernorm.weight
                                                                  
[0179/0805] saving model.layers.21.mlp.down_proj.q_weight
                                                                  
[0180/0805] saving model.layers.21.mlp.down_proj.q_scale
                                                                  
[0181/0805] saving model.layers.21.mlp.gate_up_proj.q_weight
                                                                  
[0182/0805] saving model.layers.21.mlp.gate_up_proj.q_scale
                                                                  
[0183/0805] saving model.layers.21.post_attention_layernorm.weight
                                                                  
[0184/0805] saving model.layers.21.self_attn.qkv_proj.q_weight
                                                                  
[0185/0805] saving model.layers.21.self_attn.qkv_proj.q_scale
                                                                  
[0186/0805] saving model.layers.21.self_attn.o_proj.q_weight
                                                                  
[0187/0805] saving model.layers.21.self_attn.o_proj.q_scale
                                                                  
[0188/0805] saving model.layers.22.self_attn.qkv_proj.q_weight
                                                                  
[0189/0805] saving model.layers.22.self_attn.qkv_proj.q_scale
                                                                  
[0190/0805] saving model.layers.2.input_layernorm.weight
                                                                  
[0191/0805] saving model.layers.2.mlp.down_proj.q_weight
                                                                  
[0192/0805] saving model.layers.2.mlp.down_proj.q_scale
                                                                  
[0193/0805] saving model.layers.2.post_attention_layernorm.weight
                                                                  
[0194/0805] saving model.layers.3.input_layernorm.weight
                                                                  
[0195/0805] saving model.layers.3.mlp.down_proj.q_weight
                                                                  
[0196/0805] saving model.layers.3.mlp.down_proj.q_scale
                                                                  
[0197/0805] saving model.layers.3.mlp.gate_up_proj.q_weight
                                                                  
[0198/0805] saving model.layers.3.mlp.gate_up_proj.q_scale
                                                                  
[0199/0805] saving model.layers.3.post_attention_layernorm.weight
                                                                  
[0200/0805] saving model.layers.3.self_attn.qkv_proj.q_weight
                                                                  
[0201/0805] saving model.layers.3.self_attn.qkv_proj.q_scale
                                                                  
[0202/0805] saving model.layers.3.self_attn.o_proj.q_weight
                                                                  
[0203/0805] saving model.layers.3.self_attn.o_proj.q_scale
                                                                  
[0204/0805] saving model.layers.4.input_layernorm.weight
                                                                  
[0205/0805] saving model.layers.4.mlp.down_proj.q_weight
                                                                  
[0206/0805] saving model.layers.4.mlp.down_proj.q_scale
                                                                  
[0207/0805] saving model.layers.4.mlp.gate_up_proj.q_weight
                                                                  
[0208/0805] saving model.layers.4.mlp.gate_up_proj.q_scale
                                                                  
[0209/0805] saving model.layers.4.post_attention_layernorm.weight
                                                                  
[0210/0805] saving model.layers.4.self_attn.qkv_proj.q_weight
                                                                  
[0211/0805] saving model.layers.4.self_attn.qkv_proj.q_scale
                                                                  
[0212/0805] saving model.layers.4.self_attn.o_proj.q_weight
                                                                  
[0213/0805] saving model.layers.4.self_attn.o_proj.q_scale
                                                                  
[0214/0805] saving model.layers.5.self_attn.qkv_proj.q_weight
                                                                  
[0215/0805] saving model.layers.5.self_attn.qkv_proj.q_scale
                                                                  
[0216/0805] saving model.layers.5.self_attn.o_proj.q_weight
                                                                  
[0217/0805] saving model.layers.5.self_attn.o_proj.q_scale
                                                                  
[0218/0805] saving model.layers.22.input_layernorm.weight
                                                                  
[0219/0805] saving model.layers.22.mlp.down_proj.q_weight
                                                                  
[0220/0805] saving model.layers.22.mlp.down_proj.q_scale
                                                                  
[0221/0805] saving model.layers.22.mlp.gate_up_proj.q_weight
                                                                  
[0222/0805] saving model.layers.22.mlp.gate_up_proj.q_scale
                                                                  
[0223/0805] saving model.layers.22.post_attention_layernorm.weight
                                                                  
[0224/0805] saving model.layers.22.self_attn.o_proj.q_weight
                                                                  
[0225/0805] saving model.layers.22.self_attn.o_proj.q_scale
                                                                  
[0226/0805] saving model.layers.23.input_layernorm.weight
                                                                  
[0227/0805] saving model.layers.23.mlp.down_proj.q_weight
                                                                  
[0228/0805] saving model.layers.23.mlp.down_proj.q_scale
                                                                  
[0229/0805] saving model.layers.23.mlp.gate_up_proj.q_weight
                                                                  
[0230/0805] saving model.layers.23.mlp.gate_up_proj.q_scale
                                                                  
[0231/0805] saving model.layers.23.post_attention_layernorm.weight
                                                                  
[0232/0805] saving model.layers.23.self_attn.qkv_proj.q_weight
                                                                  
[0233/0805] saving model.layers.23.self_attn.qkv_proj.q_scale
                                                                  
[0234/0805] saving model.layers.23.self_attn.o_proj.q_weight
                                                                  
[0235/0805] saving model.layers.23.self_attn.o_proj.q_scale
                                                                  
[0236/0805] saving model.layers.24.input_layernorm.weight
                                                                  
[0237/0805] saving model.layers.24.mlp.down_proj.q_weight
                                                                  
[0238/0805] saving model.layers.24.mlp.down_proj.q_scale
                                                                  
[0239/0805] saving model.layers.24.mlp.gate_up_proj.q_weight
                                                                  
[0240/0805] saving model.layers.24.mlp.gate_up_proj.q_scale
                                                                  
[0241/0805] saving model.layers.24.post_attention_layernorm.weight
                                                                  
[0242/0805] saving model.layers.24.self_attn.qkv_proj.q_weight
                                                                  
[0243/0805] saving model.layers.24.self_attn.qkv_proj.q_scale
                                                                  
[0244/0805] saving model.layers.24.self_attn.o_proj.q_weight
                                                                  
[0245/0805] saving model.layers.24.self_attn.o_proj.q_scale
                                                                  
[0246/0805] saving model.layers.25.input_layernorm.weight
                                                                  
[0247/0805] saving model.layers.25.mlp.down_proj.q_weight
                                                                  
[0248/0805] saving model.layers.25.mlp.down_proj.q_scale
                                                                  
[0249/0805] saving model.layers.25.mlp.gate_up_proj.q_weight
                                                                  
[0250/0805] saving model.layers.25.mlp.gate_up_proj.q_scale
                                                                  
[0251/0805] saving model.layers.25.post_attention_layernorm.weight
                                                                  
[0252/0805] saving model.layers.25.self_attn.qkv_proj.q_weight
                                                                  
[0253/0805] saving model.layers.25.self_attn.qkv_proj.q_scale
                                                                  
[0254/0805] saving model.layers.25.self_attn.o_proj.q_weight
                                                                  
[0255/0805] saving model.layers.25.self_attn.o_proj.q_scale
                                                                  
[0256/0805] saving model.layers.26.input_layernorm.weight
                                                                  
[0257/0805] saving model.layers.26.mlp.down_proj.q_weight
                                                                  
[0258/0805] saving model.layers.26.mlp.down_proj.q_scale
                                                                  
[0259/0805] saving model.layers.26.mlp.gate_up_proj.q_weight
                                                                  
[0260/0805] saving model.layers.26.mlp.gate_up_proj.q_scale
                                                                  
[0261/0805] saving model.layers.26.post_attention_layernorm.weight
                                                                  
[0262/0805] saving model.layers.26.self_attn.qkv_proj.q_weight
                                                                  
[0263/0805] saving model.layers.26.self_attn.qkv_proj.q_scale
                                                                  
[0264/0805] saving model.layers.26.self_attn.o_proj.q_weight
                                                                  
[0265/0805] saving model.layers.26.self_attn.o_proj.q_scale
                                                                  
[0266/0805] saving model.layers.27.mlp.gate_up_proj.q_weight
                                                                  
[0267/0805] saving model.layers.27.mlp.gate_up_proj.q_scale
                                                                  
[0268/0805] saving model.layers.27.self_attn.qkv_proj.q_weight
                                                                  
[0269/0805] saving model.layers.27.self_attn.qkv_proj.q_scale
                                                                  
[0270/0805] saving model.layers.27.self_attn.o_proj.q_weight
                                                                  
[0271/0805] saving model.layers.27.self_attn.o_proj.q_scale
                                                                  
[0272/0805] saving model.layers.27.input_layernorm.weight
                                                                  
[0273/0805] saving model.layers.27.mlp.down_proj.q_weight
                                                                  
[0274/0805] saving model.layers.27.mlp.down_proj.q_scale
                                                                  
[0275/0805] saving model.layers.27.post_attention_layernorm.weight
                                                                  
[0276/0805] saving model.layers.28.input_layernorm.weight
                                                                  
[0277/0805] saving model.layers.28.mlp.down_proj.q_weight
                                                                  
[0278/0805] saving model.layers.28.mlp.down_proj.q_scale
                                                                  
[0279/0805] saving model.layers.28.mlp.gate_up_proj.q_weight
                                                                  
[0280/0805] saving model.layers.28.mlp.gate_up_proj.q_scale
                                                                  
[0281/0805] saving model.layers.28.post_attention_layernorm.weight
                                                                  
[0282/0805] saving model.layers.28.self_attn.qkv_proj.q_weight
                                                                  
[0283/0805] saving model.layers.28.self_attn.qkv_proj.q_scale
                                                                  
[0284/0805] saving model.layers.28.self_attn.o_proj.q_weight
                                                                  
[0285/0805] saving model.layers.28.self_attn.o_proj.q_scale
                                                                  
[0286/0805] saving model.layers.29.input_layernorm.weight
                                                                  
[0287/0805] saving model.layers.29.mlp.down_proj.q_weight
                                                                  
[0288/0805] saving model.layers.29.mlp.down_proj.q_scale
                                                                  
[0289/0805] saving model.layers.29.mlp.gate_up_proj.q_weight
                                                                  
[0290/0805] saving model.layers.29.mlp.gate_up_proj.q_scale
                                                                  
[0291/0805] saving model.layers.29.post_attention_layernorm.weight
                                                                  
[0292/0805] saving model.layers.29.self_attn.qkv_proj.q_weight
                                                                  
[0293/0805] saving model.layers.29.self_attn.qkv_proj.q_scale
                                                                  
[0294/0805] saving model.layers.29.self_attn.o_proj.q_weight
                                                                  
[0295/0805] saving model.layers.29.self_attn.o_proj.q_scale
                                                                  
[0296/0805] saving model.layers.30.mlp.gate_up_proj.q_weight
                                                                  
[0297/0805] saving model.layers.30.mlp.gate_up_proj.q_scale
                                                                  
[0298/0805] saving model.layers.30.self_attn.qkv_proj.q_weight
                                                                  
[0299/0805] saving model.layers.30.self_attn.qkv_proj.q_scale
                                                                  
[0300/0805] saving model.layers.30.self_attn.o_proj.q_weight
                                                                  
[0301/0805] saving model.layers.30.self_attn.o_proj.q_scale
                                                                  
[0302/0805] saving model.layers.30.input_layernorm.weight
                                                                  
[0303/0805] saving model.layers.30.mlp.down_proj.q_weight
                                                                  
[0304/0805] saving model.layers.30.mlp.down_proj.q_scale
                                                                  
[0305/0805] saving model.layers.30.post_attention_layernorm.weight
                                                                  
[0306/0805] saving model.layers.31.input_layernorm.weight
                                                                  
[0307/0805] saving model.layers.31.mlp.down_proj.q_weight
                                                                  
[0308/0805] saving model.layers.31.mlp.down_proj.q_scale
                                                                  
[0309/0805] saving model.layers.31.mlp.gate_up_proj.q_weight
                                                                  
[0310/0805] saving model.layers.31.mlp.gate_up_proj.q_scale
                                                                  
[0311/0805] saving model.layers.31.post_attention_layernorm.weight
                                                                  
[0312/0805] saving model.layers.31.self_attn.qkv_proj.q_weight
                                                                  
[0313/0805] saving model.layers.31.self_attn.qkv_proj.q_scale
                                                                  
[0314/0805] saving model.layers.31.self_attn.o_proj.q_weight
                                                                  
[0315/0805] saving model.layers.31.self_attn.o_proj.q_scale
                                                                  
[0316/0805] saving model.layers.32.input_layernorm.weight
                                                                  
[0317/0805] saving model.layers.32.mlp.down_proj.q_weight
                                                                  
[0318/0805] saving model.layers.32.mlp.down_proj.q_scale
                                                                  
[0319/0805] saving model.layers.32.mlp.gate_up_proj.q_weight
                                                                  
[0320/0805] saving model.layers.32.mlp.gate_up_proj.q_scale
                                                                  
[0321/0805] saving model.layers.32.post_attention_layernorm.weight
                                                                  
[0322/0805] saving model.layers.32.self_attn.qkv_proj.q_weight
                                                                  
[0323/0805] saving model.layers.32.self_attn.qkv_proj.q_scale
                                                                  
[0324/0805] saving model.layers.32.self_attn.o_proj.q_weight
                                                                  
[0325/0805] saving model.layers.32.self_attn.o_proj.q_scale
                                                                  
[0326/0805] saving model.layers.33.self_attn.qkv_proj.q_weight
                                                                  
[0327/0805] saving model.layers.33.self_attn.qkv_proj.q_scale
                                                                  
[0328/0805] saving model.layers.33.self_attn.o_proj.q_weight
                                                                  
[0329/0805] saving model.layers.33.self_attn.o_proj.q_scale
                                                                  
[0330/0805] saving model.layers.33.input_layernorm.weight
                                                                  
[0331/0805] saving model.layers.33.mlp.down_proj.q_weight
                                                                  
[0332/0805] saving model.layers.33.mlp.down_proj.q_scale
                                                                  
[0333/0805] saving model.layers.33.mlp.gate_up_proj.q_weight
                                                                  
[0334/0805] saving model.layers.33.mlp.gate_up_proj.q_scale
                                                                  
[0335/0805] saving model.layers.33.post_attention_layernorm.weight
                                                                  
[0336/0805] saving model.layers.34.input_layernorm.weight
                                                                  
[0337/0805] saving model.layers.34.mlp.down_proj.q_weight
                                                                  
[0338/0805] saving model.layers.34.mlp.down_proj.q_scale
                                                                  
[0339/0805] saving model.layers.34.mlp.gate_up_proj.q_weight
                                                                  
[0340/0805] saving model.layers.34.mlp.gate_up_proj.q_scale
                                                                  
[0341/0805] saving model.layers.34.post_attention_layernorm.weight
                                                                  
[0342/0805] saving model.layers.34.self_attn.qkv_proj.q_weight
                                                                  
[0343/0805] saving model.layers.34.self_attn.qkv_proj.q_scale
                                                                  
[0344/0805] saving model.layers.34.self_attn.o_proj.q_weight
                                                                  
[0345/0805] saving model.layers.34.self_attn.o_proj.q_scale
                                                                  
[0346/0805] saving model.layers.35.input_layernorm.weight
                                                                  
[0347/0805] saving model.layers.35.mlp.down_proj.q_weight
                                                                  
[0348/0805] saving model.layers.35.mlp.down_proj.q_scale
                                                                  
[0349/0805] saving model.layers.35.mlp.gate_up_proj.q_weight
                                                                  
[0350/0805] saving model.layers.35.mlp.gate_up_proj.q_scale
                                                                  
[0351/0805] saving model.layers.35.post_attention_layernorm.weight
                                                                  
[0352/0805] saving model.layers.35.self_attn.qkv_proj.q_weight
                                                                  
[0353/0805] saving model.layers.35.self_attn.qkv_proj.q_scale
                                                                  
[0354/0805] saving model.layers.35.self_attn.o_proj.q_weight
                                                                  
[0355/0805] saving model.layers.35.self_attn.o_proj.q_scale
                                                                  
[0356/0805] saving model.layers.36.self_attn.qkv_proj.q_weight
                                                                  
[0357/0805] saving model.layers.36.self_attn.qkv_proj.q_scale
                                                                  
[0358/0805] saving model.layers.36.input_layernorm.weight
                                                                  
[0359/0805] saving model.layers.36.mlp.down_proj.q_weight
                                                                  
[0360/0805] saving model.layers.36.mlp.down_proj.q_scale
                                                                  
[0361/0805] saving model.layers.36.mlp.gate_up_proj.q_weight
                                                                  
[0362/0805] saving model.layers.36.mlp.gate_up_proj.q_scale
                                                                  
[0363/0805] saving model.layers.36.post_attention_layernorm.weight
                                                                  
[0364/0805] saving model.layers.36.self_attn.o_proj.q_weight
                                                                  
[0365/0805] saving model.layers.36.self_attn.o_proj.q_scale
                                                                  
[0366/0805] saving model.layers.37.input_layernorm.weight
                                                                  
[0367/0805] saving model.layers.37.mlp.down_proj.q_weight
                                                                  
[0368/0805] saving model.layers.37.mlp.down_proj.q_scale
                                                                  
[0369/0805] saving model.layers.37.mlp.gate_up_proj.q_weight
                                                                  
[0370/0805] saving model.layers.37.mlp.gate_up_proj.q_scale
                                                                  
[0371/0805] saving model.layers.37.post_attention_layernorm.weight
                                                                  
[0372/0805] saving model.layers.37.self_attn.qkv_proj.q_weight
                                                                  
[0373/0805] saving model.layers.37.self_attn.qkv_proj.q_scale
                                                                  
[0374/0805] saving model.layers.37.self_attn.o_proj.q_weight
                                                                  
[0375/0805] saving model.layers.37.self_attn.o_proj.q_scale
                                                                  
[0376/0805] saving model.layers.38.input_layernorm.weight
                                                                  
[0377/0805] saving model.layers.38.mlp.down_proj.q_weight
                                                                  
[0378/0805] saving model.layers.38.mlp.down_proj.q_scale
                                                                  
[0379/0805] saving model.layers.38.mlp.gate_up_proj.q_weight
                                                                  
[0380/0805] saving model.layers.38.mlp.gate_up_proj.q_scale
                                                                  
[0381/0805] saving model.layers.38.post_attention_layernorm.weight
                                                                  
[0382/0805] saving model.layers.38.self_attn.qkv_proj.q_weight
                                                                  
[0383/0805] saving model.layers.38.self_attn.qkv_proj.q_scale
                                                                  
[0384/0805] saving model.layers.38.self_attn.o_proj.q_weight
                                                                  
[0385/0805] saving model.layers.38.self_attn.o_proj.q_scale
                                                                  
[0386/0805] saving model.layers.39.input_layernorm.weight
                                                                  
[0387/0805] saving model.layers.39.mlp.down_proj.q_weight
                                                                  
[0388/0805] saving model.layers.39.mlp.down_proj.q_scale
                                                                  
[0389/0805] saving model.layers.39.mlp.gate_up_proj.q_weight
                                                                  
[0390/0805] saving model.layers.39.mlp.gate_up_proj.q_scale
                                                                  
[0391/0805] saving model.layers.39.post_attention_layernorm.weight
                                                                  
[0392/0805] saving model.layers.39.self_attn.qkv_proj.q_weight
                                                                  
[0393/0805] saving model.layers.39.self_attn.qkv_proj.q_scale
                                                                  
[0394/0805] saving model.layers.39.self_attn.o_proj.q_weight
                                                                  
[0395/0805] saving model.layers.39.self_attn.o_proj.q_scale
                                                                  
[0396/0805] saving model.layers.40.input_layernorm.weight
                                                                  
[0397/0805] saving model.layers.40.mlp.down_proj.q_weight
                                                                  
[0398/0805] saving model.layers.40.mlp.down_proj.q_scale
                                                                  
[0399/0805] saving model.layers.40.mlp.gate_up_proj.q_weight
                                                                  
[0400/0805] saving model.layers.40.mlp.gate_up_proj.q_scale
                                                                  
[0401/0805] saving model.layers.40.post_attention_layernorm.weight
                                                                  
[0402/0805] saving model.layers.40.self_attn.qkv_proj.q_weight
                                                                  
[0403/0805] saving model.layers.40.self_attn.qkv_proj.q_scale
                                                                  
[0404/0805] saving model.layers.40.self_attn.o_proj.q_weight
                                                                  
[0405/0805] saving model.layers.40.self_attn.o_proj.q_scale
                                                                  
[0406/0805] saving model.layers.41.mlp.gate_up_proj.q_weight
                                                                  
[0407/0805] saving model.layers.41.mlp.gate_up_proj.q_scale
                                                                  
[0408/0805] saving model.layers.41.self_attn.qkv_proj.q_weight
                                                                  
[0409/0805] saving model.layers.41.self_attn.qkv_proj.q_scale
                                                                  
[0410/0805] saving model.layers.41.self_attn.o_proj.q_weight
                                                                  
[0411/0805] saving model.layers.41.self_attn.o_proj.q_scale
                                                                  
[0412/0805] saving model.layers.41.input_layernorm.weight
                                                                  
[0413/0805] saving model.layers.41.mlp.down_proj.q_weight
                                                                  
[0414/0805] saving model.layers.41.mlp.down_proj.q_scale
                                                                  
[0415/0805] saving model.layers.41.post_attention_layernorm.weight
                                                                  
[0416/0805] saving model.layers.42.input_layernorm.weight
                                                                  
[0417/0805] saving model.layers.42.mlp.down_proj.q_weight
                                                                  
[0418/0805] saving model.layers.42.mlp.down_proj.q_scale
                                                                  
[0419/0805] saving model.layers.42.mlp.gate_up_proj.q_weight
                                                                  
[0420/0805] saving model.layers.42.mlp.gate_up_proj.q_scale
                                                                  
[0421/0805] saving model.layers.42.post_attention_layernorm.weight
                                                                  
[0422/0805] saving model.layers.42.self_attn.qkv_proj.q_weight
                                                                  
[0423/0805] saving model.layers.42.self_attn.qkv_proj.q_scale
                                                                  
[0424/0805] saving model.layers.42.self_attn.o_proj.q_weight
                                                                  
[0425/0805] saving model.layers.42.self_attn.o_proj.q_scale
                                                                  
[0426/0805] saving model.layers.43.input_layernorm.weight
                                                                  
[0427/0805] saving model.layers.43.mlp.down_proj.q_weight
                                                                  
[0428/0805] saving model.layers.43.mlp.down_proj.q_scale
                                                                  
[0429/0805] saving model.layers.43.mlp.gate_up_proj.q_weight
                                                                  
[0430/0805] saving model.layers.43.mlp.gate_up_proj.q_scale
                                                                  
[0431/0805] saving model.layers.43.post_attention_layernorm.weight
                                                                  
[0432/0805] saving model.layers.43.self_attn.qkv_proj.q_weight
                                                                  
[0433/0805] saving model.layers.43.self_attn.qkv_proj.q_scale
                                                                  
[0434/0805] saving model.layers.43.self_attn.o_proj.q_weight
                                                                  
[0435/0805] saving model.layers.43.self_attn.o_proj.q_scale
                                                                  
[0436/0805] saving model.layers.44.mlp.gate_up_proj.q_weight
                                                                  
[0437/0805] saving model.layers.44.mlp.gate_up_proj.q_scale
                                                                  
[0438/0805] saving model.layers.44.self_attn.qkv_proj.q_weight
                                                                  
[0439/0805] saving model.layers.44.self_attn.qkv_proj.q_scale
                                                                  
[0440/0805] saving model.layers.44.self_attn.o_proj.q_weight
                                                                  
[0441/0805] saving model.layers.44.self_attn.o_proj.q_scale
                                                                  
[0442/0805] saving model.layers.44.input_layernorm.weight
                                                                  
[0443/0805] saving model.layers.44.mlp.down_proj.q_weight
                                                                  
[0444/0805] saving model.layers.44.mlp.down_proj.q_scale
                                                                  
[0445/0805] saving model.layers.44.post_attention_layernorm.weight
                                                                  
[0446/0805] saving model.layers.45.input_layernorm.weight
                                                                  
[0447/0805] saving model.layers.45.mlp.down_proj.q_weight
                                                                  
[0448/0805] saving model.layers.45.mlp.down_proj.q_scale
                                                                  
[0449/0805] saving model.layers.45.mlp.gate_up_proj.q_weight
                                                                  
[0450/0805] saving model.layers.45.mlp.gate_up_proj.q_scale
                                                                  
[0451/0805] saving model.layers.45.post_attention_layernorm.weight
                                                                  
[0452/0805] saving model.layers.45.self_attn.qkv_proj.q_weight
                                                                  
[0453/0805] saving model.layers.45.self_attn.qkv_proj.q_scale
                                                                  
[0454/0805] saving model.layers.45.self_attn.o_proj.q_weight
                                                                  
[0455/0805] saving model.layers.45.self_attn.o_proj.q_scale
                                                                  
[0456/0805] saving model.layers.46.input_layernorm.weight
                                                                  
[0457/0805] saving model.layers.46.mlp.down_proj.q_weight
                                                                  
[0458/0805] saving model.layers.46.mlp.down_proj.q_scale
                                                                  
[0459/0805] saving model.layers.46.mlp.gate_up_proj.q_weight
                                                                  
[0460/0805] saving model.layers.46.mlp.gate_up_proj.q_scale
                                                                  
[0461/0805] saving model.layers.46.post_attention_layernorm.weight
                                                                  
[0462/0805] saving model.layers.46.self_attn.qkv_proj.q_weight
                                                                  
[0463/0805] saving model.layers.46.self_attn.qkv_proj.q_scale
                                                                  
[0464/0805] saving model.layers.46.self_attn.o_proj.q_weight
                                                                  
[0465/0805] saving model.layers.46.self_attn.o_proj.q_scale
                                                                  
[0466/0805] saving model.layers.47.self_attn.qkv_proj.q_weight
                                                                  
[0467/0805] saving model.layers.47.self_attn.qkv_proj.q_scale
                                                                  
[0468/0805] saving model.layers.47.self_attn.o_proj.q_weight
                                                                  
[0469/0805] saving model.layers.47.self_attn.o_proj.q_scale
                                                                  
[0470/0805] saving model.layers.47.input_layernorm.weight
                                                                  
[0471/0805] saving model.layers.47.mlp.down_proj.q_weight
                                                                  
[0472/0805] saving model.layers.47.mlp.down_proj.q_scale
                                                                  
[0473/0805] saving model.layers.47.mlp.gate_up_proj.q_weight
                                                                  
[0474/0805] saving model.layers.47.mlp.gate_up_proj.q_scale
                                                                  
[0475/0805] saving model.layers.47.post_attention_layernorm.weight
                                                                  
[0476/0805] saving model.layers.48.input_layernorm.weight
                                                                  
[0477/0805] saving model.layers.48.mlp.down_proj.q_weight
                                                                  
[0478/0805] saving model.layers.48.mlp.down_proj.q_scale
                                                                  
[0479/0805] saving model.layers.48.mlp.gate_up_proj.q_weight
                                                                  
[0480/0805] saving model.layers.48.mlp.gate_up_proj.q_scale
                                                                  
[0481/0805] saving model.layers.48.post_attention_layernorm.weight
                                                                  
[0482/0805] saving model.layers.48.self_attn.qkv_proj.q_weight
                                                                  
[0483/0805] saving model.layers.48.self_attn.qkv_proj.q_scale
                                                                  
[0484/0805] saving model.layers.48.self_attn.o_proj.q_weight
                                                                  
[0485/0805] saving model.layers.48.self_attn.o_proj.q_scale
                                                                  
[0486/0805] saving model.layers.49.input_layernorm.weight
                                                                  
[0487/0805] saving model.layers.49.mlp.down_proj.q_weight
                                                                  
[0488/0805] saving model.layers.49.mlp.down_proj.q_scale
                                                                  
[0489/0805] saving model.layers.49.mlp.gate_up_proj.q_weight
                                                                  
[0490/0805] saving model.layers.49.mlp.gate_up_proj.q_scale
                                                                  
[0491/0805] saving model.layers.49.post_attention_layernorm.weight
                                                                  
[0492/0805] saving model.layers.49.self_attn.qkv_proj.q_weight
                                                                  
[0493/0805] saving model.layers.49.self_attn.qkv_proj.q_scale
                                                                  
[0494/0805] saving model.layers.49.self_attn.o_proj.q_weight
                                                                  
[0495/0805] saving model.layers.49.self_attn.o_proj.q_scale
                                                                  
[0496/0805] saving model.layers.50.self_attn.qkv_proj.q_weight
                                                                  
[0497/0805] saving model.layers.50.self_attn.qkv_proj.q_scale
                                                                  
[0498/0805] saving model.layers.5.input_layernorm.weight
                                                                  
[0499/0805] saving model.layers.5.mlp.down_proj.q_weight
                                                                  
[0500/0805] saving model.layers.5.mlp.down_proj.q_scale
                                                                  
[0501/0805] saving model.layers.5.mlp.gate_up_proj.q_weight
                                                                  
[0502/0805] saving model.layers.5.mlp.gate_up_proj.q_scale
                                                                  
[0503/0805] saving model.layers.5.post_attention_layernorm.weight
                                                                  
[0504/0805] saving model.layers.6.input_layernorm.weight
                                                                  
[0505/0805] saving model.layers.6.mlp.down_proj.q_weight
                                                                  
[0506/0805] saving model.layers.6.mlp.down_proj.q_scale
                                                                  
[0507/0805] saving model.layers.6.mlp.gate_up_proj.q_weight
                                                                  
[0508/0805] saving model.layers.6.mlp.gate_up_proj.q_scale
                                                                  
[0509/0805] saving model.layers.6.post_attention_layernorm.weight
                                                                  
[0510/0805] saving model.layers.6.self_attn.qkv_proj.q_weight
                                                                  
[0511/0805] saving model.layers.6.self_attn.qkv_proj.q_scale
                                                                  
[0512/0805] saving model.layers.6.self_attn.o_proj.q_weight
                                                                  
[0513/0805] saving model.layers.6.self_attn.o_proj.q_scale
                                                                  
[0514/0805] saving model.layers.7.input_layernorm.weight
                                                                  
[0515/0805] saving model.layers.7.mlp.down_proj.q_weight
                                                                  
[0516/0805] saving model.layers.7.mlp.down_proj.q_scale
                                                                  
[0517/0805] saving model.layers.7.mlp.gate_up_proj.q_weight
                                                                  
[0518/0805] saving model.layers.7.mlp.gate_up_proj.q_scale
                                                                  
[0519/0805] saving model.layers.7.post_attention_layernorm.weight
                                                                  
[0520/0805] saving model.layers.7.self_attn.qkv_proj.q_weight
                                                                  
[0521/0805] saving model.layers.7.self_attn.qkv_proj.q_scale
                                                                  
[0522/0805] saving model.layers.7.self_attn.o_proj.q_weight
                                                                  
[0523/0805] saving model.layers.7.self_attn.o_proj.q_scale
                                                                  
[0524/0805] saving model.layers.8.self_attn.qkv_proj.q_weight
                                                                  
[0525/0805] saving model.layers.8.self_attn.qkv_proj.q_scale
                                                                  
[0526/0805] saving model.layers.50.input_layernorm.weight
                                                                  
[0527/0805] saving model.layers.50.mlp.down_proj.q_weight
                                                                  
[0528/0805] saving model.layers.50.mlp.down_proj.q_scale
                                                                  
[0529/0805] saving model.layers.50.mlp.gate_up_proj.q_weight
                                                                  
[0530/0805] saving model.layers.50.mlp.gate_up_proj.q_scale
                                                                  
[0531/0805] saving model.layers.50.post_attention_layernorm.weight
                                                                  
[0532/0805] saving model.layers.50.self_attn.o_proj.q_weight
                                                                  
[0533/0805] saving model.layers.50.self_attn.o_proj.q_scale
                                                                  
[0534/0805] saving model.layers.51.input_layernorm.weight
                                                                  
[0535/0805] saving model.layers.51.mlp.down_proj.q_weight
                                                                  
[0536/0805] saving model.layers.51.mlp.down_proj.q_scale
                                                                  
[0537/0805] saving model.layers.51.mlp.gate_up_proj.q_weight
                                                                  
[0538/0805] saving model.layers.51.mlp.gate_up_proj.q_scale
                                                                  
[0539/0805] saving model.layers.51.post_attention_layernorm.weight
                                                                  
[0540/0805] saving model.layers.51.self_attn.qkv_proj.q_weight
                                                                  
[0541/0805] saving model.layers.51.self_attn.qkv_proj.q_scale
                                                                  
[0542/0805] saving model.layers.51.self_attn.o_proj.q_weight
                                                                  
[0543/0805] saving model.layers.51.self_attn.o_proj.q_scale
                                                                  
[0544/0805] saving model.layers.52.input_layernorm.weight
                                                                  
[0545/0805] saving model.layers.52.mlp.down_proj.q_weight
                                                                  
[0546/0805] saving model.layers.52.mlp.down_proj.q_scale
                                                                  
[0547/0805] saving model.layers.52.mlp.gate_up_proj.q_weight
                                                                  
[0548/0805] saving model.layers.52.mlp.gate_up_proj.q_scale
                                                                  
[0549/0805] saving model.layers.52.post_attention_layernorm.weight
                                                                  
[0550/0805] saving model.layers.52.self_attn.qkv_proj.q_weight
                                                                  
[0551/0805] saving model.layers.52.self_attn.qkv_proj.q_scale
                                                                  
[0552/0805] saving model.layers.52.self_attn.o_proj.q_weight
                                                                  
[0553/0805] saving model.layers.52.self_attn.o_proj.q_scale
                                                                  
[0554/0805] saving model.layers.53.input_layernorm.weight
                                                                  
[0555/0805] saving model.layers.53.mlp.down_proj.q_weight
                                                                  
[0556/0805] saving model.layers.53.mlp.down_proj.q_scale
                                                                  
[0557/0805] saving model.layers.53.mlp.gate_up_proj.q_weight
                                                                  
[0558/0805] saving model.layers.53.mlp.gate_up_proj.q_scale
                                                                  
[0559/0805] saving model.layers.53.post_attention_layernorm.weight
                                                                  
[0560/0805] saving model.layers.53.self_attn.qkv_proj.q_weight
                                                                  
[0561/0805] saving model.layers.53.self_attn.qkv_proj.q_scale
                                                                  
[0562/0805] saving model.layers.53.self_attn.o_proj.q_weight
                                                                  
[0563/0805] saving model.layers.53.self_attn.o_proj.q_scale
                                                                  
[0564/0805] saving model.layers.54.input_layernorm.weight
                                                                  
[0565/0805] saving model.layers.54.mlp.down_proj.q_weight
                                                                  
[0566/0805] saving model.layers.54.mlp.down_proj.q_scale
                                                                  
[0567/0805] saving model.layers.54.mlp.gate_up_proj.q_weight
                                                                  
[0568/0805] saving model.layers.54.mlp.gate_up_proj.q_scale
                                                                  
[0569/0805] saving model.layers.54.post_attention_layernorm.weight
                                                                  
[0570/0805] saving model.layers.54.self_attn.qkv_proj.q_weight
                                                                  
[0571/0805] saving model.layers.54.self_attn.qkv_proj.q_scale
                                                                  
[0572/0805] saving model.layers.54.self_attn.o_proj.q_weight
                                                                  
[0573/0805] saving model.layers.54.self_attn.o_proj.q_scale
                                                                  
[0574/0805] saving model.layers.55.mlp.gate_up_proj.q_weight
                                                                  
[0575/0805] saving model.layers.55.mlp.gate_up_proj.q_scale
                                                                  
[0576/0805] saving model.layers.55.self_attn.qkv_proj.q_weight
                                                                  
[0577/0805] saving model.layers.55.self_attn.qkv_proj.q_scale
                                                                  
[0578/0805] saving model.layers.55.self_attn.o_proj.q_weight
                                                                  
[0579/0805] saving model.layers.55.self_attn.o_proj.q_scale
                                                                  
[0580/0805] saving model.layers.55.input_layernorm.weight
                                                                  
[0581/0805] saving model.layers.55.mlp.down_proj.q_weight
                                                                  
[0582/0805] saving model.layers.55.mlp.down_proj.q_scale
                                                                  
[0583/0805] saving model.layers.55.post_attention_layernorm.weight
                                                                  
[0584/0805] saving model.layers.56.input_layernorm.weight
                                                                  
[0585/0805] saving model.layers.56.mlp.down_proj.q_weight
                                                                  
[0586/0805] saving model.layers.56.mlp.down_proj.q_scale
                                                                  
[0587/0805] saving model.layers.56.mlp.gate_up_proj.q_weight
                                                                  
[0588/0805] saving model.layers.56.mlp.gate_up_proj.q_scale
                                                                  
[0589/0805] saving model.layers.56.post_attention_layernorm.weight
                                                                  
[0590/0805] saving model.layers.56.self_attn.qkv_proj.q_weight
                                                                  
[0591/0805] saving model.layers.56.self_attn.qkv_proj.q_scale
                                                                  
[0592/0805] saving model.layers.56.self_attn.o_proj.q_weight
                                                                  
[0593/0805] saving model.layers.56.self_attn.o_proj.q_scale
                                                                  
[0594/0805] saving model.layers.57.input_layernorm.weight
                                                                  
[0595/0805] saving model.layers.57.mlp.down_proj.q_weight
                                                                  
[0596/0805] saving model.layers.57.mlp.down_proj.q_scale
                                                                  
[0597/0805] saving model.layers.57.mlp.gate_up_proj.q_weight
                                                                  
[0598/0805] saving model.layers.57.mlp.gate_up_proj.q_scale
                                                                  
[0599/0805] saving model.layers.57.post_attention_layernorm.weight
                                                                  
[0600/0805] saving model.layers.57.self_attn.qkv_proj.q_weight
                                                                  
[0601/0805] saving model.layers.57.self_attn.qkv_proj.q_scale
                                                                  
[0602/0805] saving model.layers.57.self_attn.o_proj.q_weight
                                                                  
[0603/0805] saving model.layers.57.self_attn.o_proj.q_scale
                                                                  
[0604/0805] saving model.layers.58.mlp.gate_up_proj.q_weight
                                                                  
[0605/0805] saving model.layers.58.mlp.gate_up_proj.q_scale
                                                                  
[0606/0805] saving model.layers.58.self_attn.qkv_proj.q_weight
                                                                  
[0607/0805] saving model.layers.58.self_attn.qkv_proj.q_scale
                                                                  
[0608/0805] saving model.layers.58.self_attn.o_proj.q_weight
                                                                  
[0609/0805] saving model.layers.58.self_attn.o_proj.q_scale
                                                                  
[0610/0805] saving model.layers.58.input_layernorm.weight
                                                                  
[0611/0805] saving model.layers.58.mlp.down_proj.q_weight
                                                                  
[0612/0805] saving model.layers.58.mlp.down_proj.q_scale
                                                                  
[0613/0805] saving model.layers.58.post_attention_layernorm.weight
                                                                  
[0614/0805] saving model.layers.59.input_layernorm.weight
                                                                  
[0615/0805] saving model.layers.59.mlp.down_proj.q_weight
                                                                  
[0616/0805] saving model.layers.59.mlp.down_proj.q_scale
                                                                  
[0617/0805] saving model.layers.59.mlp.gate_up_proj.q_weight
                                                                  
[0618/0805] saving model.layers.59.mlp.gate_up_proj.q_scale
                                                                  
[0619/0805] saving model.layers.59.post_attention_layernorm.weight
                                                                  
[0620/0805] saving model.layers.59.self_attn.qkv_proj.q_weight
                                                                  
[0621/0805] saving model.layers.59.self_attn.qkv_proj.q_scale
                                                                  
[0622/0805] saving model.layers.59.self_attn.o_proj.q_weight
                                                                  
[0623/0805] saving model.layers.59.self_attn.o_proj.q_scale
                                                                  
[0624/0805] saving model.layers.60.input_layernorm.weight
                                                                  
[0625/0805] saving model.layers.60.mlp.down_proj.q_weight
                                                                  
[0626/0805] saving model.layers.60.mlp.down_proj.q_scale
                                                                  
[0627/0805] saving model.layers.60.mlp.gate_up_proj.q_weight
                                                                  
[0628/0805] saving model.layers.60.mlp.gate_up_proj.q_scale
                                                                  
[0629/0805] saving model.layers.60.post_attention_layernorm.weight
                                                                  
[0630/0805] saving model.layers.60.self_attn.qkv_proj.q_weight
                                                                  
[0631/0805] saving model.layers.60.self_attn.qkv_proj.q_scale
                                                                  
[0632/0805] saving model.layers.60.self_attn.o_proj.q_weight
                                                                  
[0633/0805] saving model.layers.60.self_attn.o_proj.q_scale
                                                                  
[0634/0805] saving model.layers.61.self_attn.qkv_proj.q_weight
                                                                  
[0635/0805] saving model.layers.61.self_attn.qkv_proj.q_scale
                                                                  
[0636/0805] saving model.layers.61.self_attn.o_proj.q_weight
                                                                  
[0637/0805] saving model.layers.61.self_attn.o_proj.q_scale
                                                                  
[0638/0805] saving model.layers.61.input_layernorm.weight
                                                                  
[0639/0805] saving model.layers.61.mlp.down_proj.q_weight
                                                                  
[0640/0805] saving model.layers.61.mlp.down_proj.q_scale
                                                                  
[0641/0805] saving model.layers.61.mlp.gate_up_proj.q_weight
                                                                  
[0642/0805] saving model.layers.61.mlp.gate_up_proj.q_scale
                                                                  
[0643/0805] saving model.layers.61.post_attention_layernorm.weight
                                                                  
[0644/0805] saving model.layers.62.input_layernorm.weight
                                                                  
[0645/0805] saving model.layers.62.mlp.down_proj.q_weight
                                                                  
[0646/0805] saving model.layers.62.mlp.down_proj.q_scale
                                                                  
[0647/0805] saving model.layers.62.mlp.gate_up_proj.q_weight
                                                                  
[0648/0805] saving model.layers.62.mlp.gate_up_proj.q_scale
                                                                  
[0649/0805] saving model.layers.62.post_attention_layernorm.weight
                                                                  
[0650/0805] saving model.layers.62.self_attn.qkv_proj.q_weight
                                                                  
[0651/0805] saving model.layers.62.self_attn.qkv_proj.q_scale
                                                                  
[0652/0805] saving model.layers.62.self_attn.o_proj.q_weight
                                                                  
[0653/0805] saving model.layers.62.self_attn.o_proj.q_scale
                                                                  
[0654/0805] saving model.layers.63.input_layernorm.weight
                                                                  
[0655/0805] saving model.layers.63.mlp.down_proj.q_weight
                                                                  
[0656/0805] saving model.layers.63.mlp.down_proj.q_scale
                                                                  
[0657/0805] saving model.layers.63.mlp.gate_up_proj.q_weight
                                                                  
[0658/0805] saving model.layers.63.mlp.gate_up_proj.q_scale
                                                                  
[0659/0805] saving model.layers.63.post_attention_layernorm.weight
                                                                  
[0660/0805] saving model.layers.63.self_attn.qkv_proj.q_weight
                                                                  
[0661/0805] saving model.layers.63.self_attn.qkv_proj.q_scale
                                                                  
[0662/0805] saving model.layers.63.self_attn.o_proj.q_weight
                                                                  
[0663/0805] saving model.layers.63.self_attn.o_proj.q_scale
                                                                  
[0664/0805] saving model.layers.64.self_attn.qkv_proj.q_weight
                                                                  
[0665/0805] saving model.layers.64.self_attn.qkv_proj.q_scale
                                                                  
[0666/0805] saving model.layers.64.input_layernorm.weight
                                                                  
[0667/0805] saving model.layers.64.mlp.down_proj.q_weight
                                                                  
[0668/0805] saving model.layers.64.mlp.down_proj.q_scale
                                                                  
[0669/0805] saving model.layers.64.mlp.gate_up_proj.q_weight
                                                                  
[0670/0805] saving model.layers.64.mlp.gate_up_proj.q_scale
                                                                  
[0671/0805] saving model.layers.64.post_attention_layernorm.weight
                                                                  
[0672/0805] saving model.layers.64.self_attn.o_proj.q_weight
                                                                  
[0673/0805] saving model.layers.64.self_attn.o_proj.q_scale
                                                                  
[0674/0805] saving model.layers.65.input_layernorm.weight
                                                                  
[0675/0805] saving model.layers.65.mlp.down_proj.q_weight
                                                                  
[0676/0805] saving model.layers.65.mlp.down_proj.q_scale
                                                                  
[0677/0805] saving model.layers.65.mlp.gate_up_proj.q_weight
                                                                  
[0678/0805] saving model.layers.65.mlp.gate_up_proj.q_scale
                                                                  
[0679/0805] saving model.layers.65.post_attention_layernorm.weight
                                                                  
[0680/0805] saving model.layers.65.self_attn.qkv_proj.q_weight
                                                                  
[0681/0805] saving model.layers.65.self_attn.qkv_proj.q_scale
                                                                  
[0682/0805] saving model.layers.65.self_attn.o_proj.q_weight
                                                                  
[0683/0805] saving model.layers.65.self_attn.o_proj.q_scale
                                                                  
[0684/0805] saving model.layers.66.input_layernorm.weight
                                                                  
[0685/0805] saving model.layers.66.mlp.down_proj.q_weight
                                                                  
[0686/0805] saving model.layers.66.mlp.down_proj.q_scale
                                                                  
[0687/0805] saving model.layers.66.mlp.gate_up_proj.q_weight
                                                                  
[0688/0805] saving model.layers.66.mlp.gate_up_proj.q_scale
                                                                  
[0689/0805] saving model.layers.66.post_attention_layernorm.weight
                                                                  
[0690/0805] saving model.layers.66.self_attn.qkv_proj.q_weight
                                                                  
[0691/0805] saving model.layers.66.self_attn.qkv_proj.q_scale
                                                                  
[0692/0805] saving model.layers.66.self_attn.o_proj.q_weight
                                                                  
[0693/0805] saving model.layers.66.self_attn.o_proj.q_scale
                                                                  
[0694/0805] saving model.layers.67.input_layernorm.weight
                                                                  
[0695/0805] saving model.layers.67.mlp.down_proj.q_weight
                                                                  
[0696/0805] saving model.layers.67.mlp.down_proj.q_scale
                                                                  
[0697/0805] saving model.layers.67.mlp.gate_up_proj.q_weight
                                                                  
[0698/0805] saving model.layers.67.mlp.gate_up_proj.q_scale
                                                                  
[0699/0805] saving model.layers.67.post_attention_layernorm.weight
                                                                  
[0700/0805] saving model.layers.67.self_attn.qkv_proj.q_weight
                                                                  
[0701/0805] saving model.layers.67.self_attn.qkv_proj.q_scale
                                                                  
[0702/0805] saving model.layers.67.self_attn.o_proj.q_weight
                                                                  
[0703/0805] saving model.layers.67.self_attn.o_proj.q_scale
                                                                  
[0704/0805] saving model.layers.68.input_layernorm.weight
                                                                  
[0705/0805] saving model.layers.68.mlp.down_proj.q_weight
                                                                  
[0706/0805] saving model.layers.68.mlp.down_proj.q_scale
                                                                  
[0707/0805] saving model.layers.68.mlp.gate_up_proj.q_weight
                                                                  
[0708/0805] saving model.layers.68.mlp.gate_up_proj.q_scale
                                                                  
[0709/0805] saving model.layers.68.post_attention_layernorm.weight
                                                                  
[0710/0805] saving model.layers.68.self_attn.qkv_proj.q_weight
                                                                  
[0711/0805] saving model.layers.68.self_attn.qkv_proj.q_scale
                                                                  
[0712/0805] saving model.layers.68.self_attn.o_proj.q_weight
                                                                  
[0713/0805] saving model.layers.68.self_attn.o_proj.q_scale
                                                                  
[0714/0805] saving model.layers.69.mlp.gate_up_proj.q_weight
                                                                  
[0715/0805] saving model.layers.69.mlp.gate_up_proj.q_scale
                                                                  
[0716/0805] saving model.layers.69.self_attn.qkv_proj.q_weight
                                                                  
[0717/0805] saving model.layers.69.self_attn.qkv_proj.q_scale
                                                                  
[0718/0805] saving model.layers.69.self_attn.o_proj.q_weight
                                                                  
[0719/0805] saving model.layers.69.self_attn.o_proj.q_scale
                                                                  
[0720/0805] saving model.layers.69.input_layernorm.weight
                                                                  
[0721/0805] saving model.layers.69.mlp.down_proj.q_weight
                                                                  
[0722/0805] saving model.layers.69.mlp.down_proj.q_scale
                                                                  
[0723/0805] saving model.layers.69.post_attention_layernorm.weight
                                                                  
[0724/0805] saving model.layers.70.input_layernorm.weight
                                                                  
[0725/0805] saving model.layers.70.mlp.down_proj.q_weight
                                                                  
[0726/0805] saving model.layers.70.mlp.down_proj.q_scale
                                                                  
[0727/0805] saving model.layers.70.mlp.gate_up_proj.q_weight
                                                                  
[0728/0805] saving model.layers.70.mlp.gate_up_proj.q_scale
                                                                  
[0729/0805] saving model.layers.70.post_attention_layernorm.weight
                                                                  
[0730/0805] saving model.layers.70.self_attn.qkv_proj.q_weight
                                                                  
[0731/0805] saving model.layers.70.self_attn.qkv_proj.q_scale
                                                                  
[0732/0805] saving model.layers.70.self_attn.o_proj.q_weight
                                                                  
[0733/0805] saving model.layers.70.self_attn.o_proj.q_scale
                                                                  
[0734/0805] saving model.layers.71.input_layernorm.weight
                                                                  
[0735/0805] saving model.layers.71.mlp.down_proj.q_weight
                                                                  
[0736/0805] saving model.layers.71.mlp.down_proj.q_scale
                                                                  
[0737/0805] saving model.layers.71.mlp.gate_up_proj.q_weight
                                                                  
[0738/0805] saving model.layers.71.mlp.gate_up_proj.q_scale
                                                                  
[0739/0805] saving model.layers.71.post_attention_layernorm.weight
                                                                  
[0740/0805] saving model.layers.71.self_attn.qkv_proj.q_weight
                                                                  
[0741/0805] saving model.layers.71.self_attn.qkv_proj.q_scale
                                                                  
[0742/0805] saving model.layers.71.self_attn.o_proj.q_weight
                                                                  
[0743/0805] saving model.layers.71.self_attn.o_proj.q_scale
                                                                  
[0744/0805] saving model.layers.72.mlp.gate_up_proj.q_weight
                                                                  
[0745/0805] saving model.layers.72.mlp.gate_up_proj.q_scale
                                                                  
[0746/0805] saving model.layers.72.self_attn.qkv_proj.q_weight
                                                                  
[0747/0805] saving model.layers.72.self_attn.qkv_proj.q_scale
                                                                  
[0748/0805] saving model.layers.72.self_attn.o_proj.q_weight
                                                                  
[0749/0805] saving model.layers.72.self_attn.o_proj.q_scale
                                                                  
[0750/0805] saving model.layers.72.input_layernorm.weight
                                                                  
[0751/0805] saving model.layers.72.mlp.down_proj.q_weight
                                                                  
[0752/0805] saving model.layers.72.mlp.down_proj.q_scale
                                                                  
[0753/0805] saving model.layers.72.post_attention_layernorm.weight
                                                                  
[0754/0805] saving model.layers.73.input_layernorm.weight
                                                                  
[0755/0805] saving model.layers.73.mlp.down_proj.q_weight
                                                                  
[0756/0805] saving model.layers.73.mlp.down_proj.q_scale
                                                                  
[0757/0805] saving model.layers.73.mlp.gate_up_proj.q_weight
                                                                  
[0758/0805] saving model.layers.73.mlp.gate_up_proj.q_scale
                                                                  
[0759/0805] saving model.layers.73.post_attention_layernorm.weight
                                                                  
[0760/0805] saving model.layers.73.self_attn.qkv_proj.q_weight
                                                                  
[0761/0805] saving model.layers.73.self_attn.qkv_proj.q_scale
                                                                  
[0762/0805] saving model.layers.73.self_attn.o_proj.q_weight
                                                                  
[0763/0805] saving model.layers.73.self_attn.o_proj.q_scale
                                                                  
[0764/0805] saving model.layers.74.input_layernorm.weight
                                                                  
[0765/0805] saving model.layers.74.mlp.down_proj.q_weight
                                                                  
[0766/0805] saving model.layers.74.mlp.down_proj.q_scale
                                                                  
[0767/0805] saving model.layers.74.mlp.gate_up_proj.q_weight
                                                                  
[0768/0805] saving model.layers.74.mlp.gate_up_proj.q_scale
                                                                  
[0769/0805] saving model.layers.74.post_attention_layernorm.weight[2024-01-30 23:28:05] INFO convert_weight.py:143: Saved to directory: /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq

                                                                  
[0770/0805] saving model.layers.74.self_attn.qkv_proj.q_weight
                                                                  
[0771/0805] saving model.layers.74.self_attn.qkv_proj.q_scale
                                                                  
[0772/0805] saving model.layers.74.self_attn.o_proj.q_weight
                                                                  
[0773/0805] saving model.layers.74.self_attn.o_proj.q_scale
                                                                  
[0774/0805] saving model.layers.75.self_attn.qkv_proj.q_weight
                                                                  
[0775/0805] saving model.layers.75.self_attn.qkv_proj.q_scale
                                                                  
[0776/0805] saving model.layers.75.self_attn.o_proj.q_weight
                                                                  
[0777/0805] saving model.layers.75.self_attn.o_proj.q_scale
                                                                  
[0778/0805] saving model.layers.75.input_layernorm.weight
                                                                  
[0779/0805] saving model.layers.75.mlp.down_proj.q_weight
                                                                  
[0780/0805] saving model.layers.75.mlp.down_proj.q_scale
                                                                  
[0781/0805] saving model.layers.75.mlp.gate_up_proj.q_weight
                                                                  
[0782/0805] saving model.layers.75.mlp.gate_up_proj.q_scale
                                                                  
[0783/0805] saving model.layers.75.post_attention_layernorm.weight
                                                                  
[0784/0805] saving model.layers.76.input_layernorm.weight
                                                                  
[0785/0805] saving model.layers.76.mlp.down_proj.q_weight
                                                                  
[0786/0805] saving model.layers.76.mlp.down_proj.q_scale
                                                                  
[0787/0805] saving model.layers.76.mlp.gate_up_proj.q_weight
                                                                  
[0788/0805] saving model.layers.76.mlp.gate_up_proj.q_scale
                                                                  
[0789/0805] saving model.layers.76.post_attention_layernorm.weight
                                                                  
[0790/0805] saving model.layers.76.self_attn.qkv_proj.q_weight
                                                                  
[0791/0805] saving model.layers.76.self_attn.qkv_proj.q_scale
                                                                  
[0792/0805] saving model.layers.76.self_attn.o_proj.q_weight
                                                                  
[0793/0805] saving model.layers.76.self_attn.o_proj.q_scale
                                                                  
[0794/0805] saving model.layers.77.input_layernorm.weight
                                                                  
[0795/0805] saving model.layers.77.mlp.down_proj.q_weight
                                                                  
[0796/0805] saving model.layers.77.mlp.down_proj.q_scale
                                                                  
[0797/0805] saving model.layers.77.mlp.gate_up_proj.q_weight
                                                                  
[0798/0805] saving model.layers.77.mlp.gate_up_proj.q_scale
                                                                  
[0799/0805] saving model.layers.77.post_attention_layernorm.weight
                                                                  
[0800/0805] saving model.layers.77.self_attn.qkv_proj.q_weight
                                                                  
[0801/0805] saving model.layers.77.self_attn.qkv_proj.q_scale
                                                                  
[0802/0805] saving model.layers.77.self_attn.o_proj.q_weight
                                                                  
[0803/0805] saving model.layers.77.self_attn.o_proj.q_scale
                                                                  
[0804/0805] saving model.layers.78.self_attn.qkv_proj.q_weight
                                                                  
[0805/0805] saving model.layers.78.self_attn.qkv_proj.q_scale
All finished, 480 total shards committed, record saved to /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq/ndarray-cache.json
Also saved a bf16 record to /var/folders/50/mzqbqxqj5fddcby2mg3h334c0000gp/T/tmpv6_x4fhq/ndarray-cache-b16.json