| /home/ruihang/Workspace/miniconda3/envs/python311/bin/python -m mlc_llm gen_config /models/Llama-2-7b-chat-hf --quantization q0f16 --conv-template llama-2 --output /tmp/tmp8ew5r2yr | |
| [2024-05-22 00:30:05] INFO auto_config.py:115: [92mFound[0m model configuration: /models/Llama-2-7b-chat-hf/config.json | |
| [2024-05-22 00:30:05] INFO auto_config.py:153: [92mFound[0m model type: [1mllama[0m. Use `--model-type` to override. | |
| [2024-05-22 00:30:05] INFO llama_model.py:52: [1mcontext_window_size[0m not found in config.json. Falling back to [1mmax_position_embeddings[0m (4096) | |
| [2024-05-22 00:30:05] INFO llama_model.py:72: [1mprefill_chunk_size[0m defaults to 2048 | |
| [2024-05-22 00:30:05] INFO config.py:106: Overriding [1mmax_batch_size[0m from 1 to 80 | |
| [2024-05-22 00:30:05] INFO gen_config.py:255: [generation_config.json] Setting [1mbos_token_id[0m: 1 | |
| [2024-05-22 00:30:05] INFO gen_config.py:255: [generation_config.json] Setting [1meos_token_id[0m: 2 | |
| [2024-05-22 00:30:05] INFO gen_config.py:255: [generation_config.json] Setting [1mpad_token_id[0m: 0 | |
| [2024-05-22 00:30:05] INFO gen_config.py:255: [generation_config.json] Setting [1mtemperature[0m: 0.6 | |
| [2024-05-22 00:30:05] INFO gen_config.py:255: [generation_config.json] Setting [1mtop_p[0m: 0.9 | |
| [2024-05-22 00:30:05] INFO gen_config.py:267: [92mFound[0m tokenizer config: /models/Llama-2-7b-chat-hf/tokenizer.model. Copying to [1m/tmp/tmp8ew5r2yr/tokenizer.model[0m | |
| [2024-05-22 00:30:05] INFO gen_config.py:267: [92mFound[0m tokenizer config: /models/Llama-2-7b-chat-hf/tokenizer.json. Copying to [1m/tmp/tmp8ew5r2yr/tokenizer.json[0m | |
| [2024-05-22 00:30:05] INFO gen_config.py:269: [91mNot found[0m tokenizer config: /models/Llama-2-7b-chat-hf/vocab.json | |
| [2024-05-22 00:30:05] INFO gen_config.py:269: [91mNot found[0m tokenizer config: /models/Llama-2-7b-chat-hf/merges.txt | |
| [2024-05-22 00:30:05] INFO gen_config.py:269: [91mNot found[0m tokenizer config: /models/Llama-2-7b-chat-hf/added_tokens.json | |
| [2024-05-22 00:30:05] INFO gen_config.py:267: [92mFound[0m tokenizer config: /models/Llama-2-7b-chat-hf/tokenizer_config.json. Copying to [1m/tmp/tmp8ew5r2yr/tokenizer_config.json[0m | |
| [2024-05-22 00:30:05] INFO gen_config.py:80: [System default] Setting [1mpresence_penalty[0m: 0.0 | |
| [2024-05-22 00:30:05] INFO gen_config.py:80: [System default] Setting [1mfrequency_penalty[0m: 0.0 | |
| [2024-05-22 00:30:05] INFO gen_config.py:80: [System default] Setting [1mrepetition_penalty[0m: 1.0 | |
| [2024-05-22 00:30:05] INFO gen_config.py:80: [System default] Setting [1mmean_gen_len[0m: 128 | |
| [2024-05-22 00:30:05] INFO gen_config.py:80: [System default] Setting [1mmax_gen_len[0m: 512 | |
| [2024-05-22 00:30:05] INFO gen_config.py:80: [System default] Setting [1mshift_fill_factor[0m: 0.3 | |
| [2024-05-22 00:30:05] INFO gen_config.py:335: Dumping configuration file to: [1m/tmp/tmp8ew5r2yr/mlc-chat-config.json[0m | |
| /home/ruihang/Workspace/miniconda3/envs/python311/bin/python -m mlc_llm convert_weight /models/Llama-2-7b-chat-hf --quantization q0f16 --source-format auto --output /tmp/tmp8ew5r2yr | |
| [2024-05-22 00:30:06] INFO auto_config.py:115: [92mFound[0m model configuration: /models/Llama-2-7b-chat-hf/config.json | |
| [2024-05-22 00:30:06] INFO auto_device.py:79: [92mFound[0m device: cuda:0 | |
| [2024-05-22 00:30:06] INFO auto_device.py:79: [92mFound[0m device: cuda:1 | |
| [2024-05-22 00:30:07] INFO auto_device.py:88: [91mNot found[0m device: rocm:0 | |
| [2024-05-22 00:30:08] INFO auto_device.py:88: [91mNot found[0m device: metal:0 | |
| [2024-05-22 00:30:09] INFO auto_device.py:79: [92mFound[0m device: vulkan:0 | |
| [2024-05-22 00:30:09] INFO auto_device.py:79: [92mFound[0m device: vulkan:1 | |
| [2024-05-22 00:30:09] INFO auto_device.py:79: [92mFound[0m device: vulkan:2 | |
| [2024-05-22 00:30:10] INFO auto_device.py:79: [92mFound[0m device: opencl:0 | |
| [2024-05-22 00:30:10] INFO auto_device.py:79: [92mFound[0m device: opencl:1 | |
| [2024-05-22 00:30:10] INFO auto_device.py:35: Using device: [1mcuda:0[0m | |
| [2024-05-22 00:30:10] INFO auto_weight.py:70: Finding weights in: /models/Llama-2-7b-chat-hf | |
| [2024-05-22 00:30:10] INFO auto_weight.py:120: [92mFound[0m source weight format: huggingface-torch. Source configuration: /models/Llama-2-7b-chat-hf/pytorch_model.bin.index.json | |
| [2024-05-22 00:30:10] INFO auto_weight.py:143: [92mFound[0m source weight format: huggingface-safetensor. Source configuration: /models/Llama-2-7b-chat-hf/model.safetensors.index.json | |
| [2024-05-22 00:30:10] INFO auto_weight.py:106: Using source weight configuration: [1m/models/Llama-2-7b-chat-hf/pytorch_model.bin.index.json[0m. Use `--source` to override. | |
| [2024-05-22 00:30:10] INFO auto_weight.py:110: Using source weight format: [1mhuggingface-torch[0m. Use `--source-format` to override. | |
| [2024-05-22 00:30:10] INFO auto_config.py:153: [92mFound[0m model type: [1mllama[0m. Use `--model-type` to override. | |
| [2024-05-22 00:30:10] INFO llama_model.py:52: [1mcontext_window_size[0m not found in config.json. Falling back to [1mmax_position_embeddings[0m (4096) | |
| [2024-05-22 00:30:10] INFO llama_model.py:72: [1mprefill_chunk_size[0m defaults to 2048 | |
| [1mWeight conversion with arguments:[0m | |
| [1m--config[0m /models/Llama-2-7b-chat-hf/config.json | |
| [1m--quantization[0m NoQuantize(name='q0f16', kind='no-quant', model_dtype='float16') | |
| [1m--model-type[0m llama | |
| [1m--device[0m cuda:0 | |
| [1m--source[0m /models/Llama-2-7b-chat-hf/pytorch_model.bin.index.json | |
| [1m--source-format[0m huggingface-torch | |
| [1m--output[0m /tmp/tmp8ew5r2yr | |
| Start storing to cache /tmp/tmp8ew5r2yr | |
| 0%| | 0/195 [00:00<?, ?it/s] [2024-05-22 00:30:11] INFO huggingface_loader.py:184: Loading HF parameters from: /models/Llama-2-7b-chat-hf/pytorch_model-00002-of-00002.bin | |
| 0%| | 0/195 [00:00<?, ?it/s] [2024-05-22 00:30:19] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mlm_head.weight[0m", shape: (32000, 4096), dtype: float16 | |
| 0%| | 0/195 [00:08<?, ?it/s] 1%| | 1/195 [00:09<29:48, 9.22s/it] [2024-05-22 00:30:20] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.24.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 1%| | 1/195 [00:09<29:48, 9.22s/it] [2024-05-22 00:30:20] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.24.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 1%| | 1/195 [00:09<29:48, 9.22s/it] 2%|▏ | 3/195 [00:09<07:58, 2.49s/it] [2024-05-22 00:30:21] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.24.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 2%|▏ | 3/195 [00:09<07:58, 2.49s/it] 2%|▏ | 4/195 [00:10<06:17, 1.98s/it] [2024-05-22 00:30:21] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.24.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 2%|▏ | 4/195 [00:10<06:17, 1.98s/it] [2024-05-22 00:30:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.24.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 2%|▏ | 4/195 [00:10<06:17, 1.98s/it] 3%|▎ | 6/195 [00:10<03:28, 1.10s/it] [2024-05-22 00:30:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.24.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 3%|▎ | 6/195 [00:10<03:28, 1.10s/it] 4%|▎ | 7/195 [00:11<02:38, 1.19it/s] [2024-05-22 00:30:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.25.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 4%|▎ | 7/195 [00:11<02:38, 1.19it/s] [2024-05-22 00:30:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.25.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 4%|▎ | 7/195 [00:11<02:38, 1.19it/s] 5%|▍ | 9/195 [00:11<01:39, 1.87it/s] [2024-05-22 00:30:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.25.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 5%|▍ | 9/195 [00:11<01:39, 1.87it/s] 5%|▌ | 10/195 [00:12<02:00, 1.54it/s] [2024-05-22 00:30:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.25.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 5%|▌ | 10/195 [00:12<02:00, 1.54it/s] [2024-05-22 00:30:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.25.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 5%|▌ | 10/195 [00:12<02:00, 1.54it/s] 6%|▌ | 12/195 [00:12<01:24, 2.15it/s] [2024-05-22 00:30:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.25.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 6%|▌ | 12/195 [00:12<01:24, 2.15it/s] 7%|▋ | 13/195 [00:12<01:09, 2.61it/s] [2024-05-22 00:30:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.26.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 7%|▋ | 13/195 [00:12<01:09, 2.61it/s] [2024-05-22 00:30:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.26.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 7%|▋ | 13/195 [00:12<01:09, 2.61it/s] 8%|▊ | 15/195 [00:13<00:50, 3.53it/s] [2024-05-22 00:30:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.26.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 8%|▊ | 15/195 [00:13<00:50, 3.53it/s] 8%|▊ | 16/195 [00:13<01:15, 2.36it/s] [2024-05-22 00:30:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.26.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 8%|▊ | 16/195 [00:13<01:15, 2.36it/s] [2024-05-22 00:30:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.26.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 8%|▊ | 16/195 [00:14<01:15, 2.36it/s] 9%|▉ | 18/195 [00:14<00:58, 3.04it/s] [2024-05-22 00:30:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.26.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 9%|▉ | 18/195 [00:14<00:58, 3.04it/s] 10%|▉ | 19/195 [00:14<00:49, 3.57it/s] [2024-05-22 00:30:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.27.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 10%|▉ | 19/195 [00:14<00:49, 3.57it/s] [2024-05-22 00:30:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.27.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 10%|▉ | 19/195 [00:14<00:49, 3.57it/s] 11%|█ | 21/195 [00:14<00:38, 4.53it/s] [2024-05-22 00:30:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.27.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 11%|█ | 21/195 [00:14<00:38, 4.53it/s] 11%|█▏ | 22/195 [00:15<00:55, 3.11it/s] [2024-05-22 00:30:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.27.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 11%|█▏ | 22/195 [00:15<00:55, 3.11it/s] [2024-05-22 00:30:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.27.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 11%|█▏ | 22/195 [00:15<00:55, 3.11it/s] 12%|█▏ | 24/195 [00:15<00:44, 3.87it/s] [2024-05-22 00:30:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.27.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 12%|█▏ | 24/195 [00:15<00:44, 3.87it/s] 13%|█▎ | 25/195 [00:15<00:38, 4.45it/s] [2024-05-22 00:30:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.28.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 13%|█▎ | 25/195 [00:15<00:38, 4.45it/s] [2024-05-22 00:30:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.28.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 13%|█▎ | 25/195 [00:15<00:38, 4.45it/s] 14%|█▍ | 27/195 [00:16<00:31, 5.38it/s] [2024-05-22 00:30:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.28.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 14%|█▍ | 27/195 [00:16<00:31, 5.38it/s] 14%|█▍ | 28/195 [00:16<00:53, 3.14it/s] [2024-05-22 00:30:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.28.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 14%|█▍ | 28/195 [00:16<00:53, 3.14it/s] [2024-05-22 00:30:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.28.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 14%|█▍ | 28/195 [00:16<00:53, 3.14it/s] 15%|█▌ | 30/195 [00:17<00:42, 3.89it/s] [2024-05-22 00:30:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.28.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 15%|█▌ | 30/195 [00:17<00:42, 3.89it/s] 16%|█▌ | 31/195 [00:17<00:36, 4.47it/s] [2024-05-22 00:30:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.29.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 16%|█▌ | 31/195 [00:17<00:36, 4.47it/s] [2024-05-22 00:30:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.29.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 16%|█▌ | 31/195 [00:17<00:36, 4.47it/s] 17%|█▋ | 33/195 [00:17<00:32, 4.97it/s] [2024-05-22 00:30:29] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.29.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 17%|█▋ | 33/195 [00:17<00:32, 4.97it/s] 17%|█▋ | 34/195 [00:18<00:51, 3.12it/s] [2024-05-22 00:30:29] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.29.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 17%|█▋ | 34/195 [00:18<00:51, 3.12it/s] [2024-05-22 00:30:29] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.29.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 17%|█▋ | 34/195 [00:18<00:51, 3.12it/s] 18%|█▊ | 36/195 [00:18<00:42, 3.77it/s] [2024-05-22 00:30:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.29.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 18%|█▊ | 36/195 [00:18<00:42, 3.77it/s] [2024-05-22 00:30:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.30.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 18%|█▊ | 36/195 [00:18<00:42, 3.77it/s] 19%|█▉ | 38/195 [00:18<00:29, 5.26it/s] [2024-05-22 00:30:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.30.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 19%|█▉ | 38/195 [00:18<00:29, 5.26it/s] 20%|██ | 39/195 [00:19<00:31, 5.02it/s] [2024-05-22 00:30:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.30.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 20%|██ | 39/195 [00:19<00:31, 5.02it/s] 21%|██ | 40/195 [00:19<00:46, 3.35it/s] [2024-05-22 00:30:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.30.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 21%|██ | 40/195 [00:19<00:46, 3.35it/s] [2024-05-22 00:30:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.30.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 21%|██ | 40/195 [00:19<00:46, 3.35it/s] 22%|██▏ | 42/195 [00:19<00:37, 4.12it/s] [2024-05-22 00:30:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.30.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 22%|██▏ | 42/195 [00:19<00:37, 4.12it/s] 22%|██▏ | 43/195 [00:20<00:32, 4.72it/s] [2024-05-22 00:30:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.31.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 22%|██▏ | 43/195 [00:20<00:32, 4.72it/s] [2024-05-22 00:30:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.31.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 22%|██▏ | 43/195 [00:20<00:32, 4.72it/s] 23%|██▎ | 45/195 [00:20<00:26, 5.62it/s] [2024-05-22 00:30:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.31.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 23%|██▎ | 45/195 [00:20<00:26, 5.62it/s] 24%|██▎ | 46/195 [00:21<00:44, 3.35it/s] [2024-05-22 00:30:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.31.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 24%|██▎ | 46/195 [00:21<00:44, 3.35it/s] [2024-05-22 00:30:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.31.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 24%|██▎ | 46/195 [00:21<00:44, 3.35it/s] 25%|██▍ | 48/195 [00:21<00:35, 4.11it/s] [2024-05-22 00:30:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.31.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 25%|██▍ | 48/195 [00:21<00:35, 4.11it/s] 25%|██▌ | 49/195 [00:21<00:31, 4.70it/s] [2024-05-22 00:30:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.norm.weight[0m", shape: (4096,), dtype: float16 | |
| 25%|██▌ | 49/195 [00:21<00:31, 4.70it/s] [2024-05-22 00:30:32] INFO huggingface_loader.py:196: Unloading HF weight file: /models/Llama-2-7b-chat-hf/pytorch_model-00002-of-00002.bin | |
| 25%|██▌ | 49/195 [00:21<00:31, 4.70it/s] [2024-05-22 00:30:32] INFO huggingface_loader.py:184: Loading HF parameters from: /models/Llama-2-7b-chat-hf/pytorch_model-00001-of-00002.bin | |
| 25%|██▌ | 49/195 [00:21<00:31, 4.70it/s] [2024-05-22 00:31:09] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.embed_tokens.weight[0m", shape: (32000, 4096), dtype: float16 | |
| 25%|██▌ | 49/195 [00:58<00:31, 4.70it/s] 26%|██▌ | 51/195 [00:58<17:18, 7.21s/it] [2024-05-22 00:31:10] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.0.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 26%|██▌ | 51/195 [00:58<17:18, 7.21s/it] [2024-05-22 00:31:10] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.0.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 26%|██▌ | 51/195 [00:58<17:18, 7.21s/it] 27%|██▋ | 53/195 [00:59<11:13, 4.74s/it] [2024-05-22 00:31:10] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.0.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 27%|██▋ | 53/195 [00:59<11:13, 4.74s/it] 28%|██▊ | 54/195 [01:00<09:23, 4.00s/it] [2024-05-22 00:31:11] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.0.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 28%|██▊ | 54/195 [01:00<09:23, 4.00s/it] [2024-05-22 00:31:11] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.0.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 28%|██▊ | 54/195 [01:00<09:23, 4.00s/it] 29%|██▊ | 56/195 [01:00<06:03, 2.61s/it] [2024-05-22 00:31:11] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.0.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 29%|██▊ | 56/195 [01:00<06:03, 2.61s/it] 29%|██▉ | 57/195 [01:00<04:50, 2.10s/it] [2024-05-22 00:31:12] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.1.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 29%|██▉ | 57/195 [01:00<04:50, 2.10s/it] [2024-05-22 00:31:12] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.1.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 29%|██▉ | 57/195 [01:00<04:50, 2.10s/it] 30%|███ | 59/195 [01:01<03:07, 1.38s/it] [2024-05-22 00:31:12] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.1.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 30%|███ | 59/195 [01:01<03:07, 1.38s/it] 31%|███ | 60/195 [01:01<02:47, 1.24s/it] [2024-05-22 00:31:13] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.1.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 31%|███ | 60/195 [01:01<02:47, 1.24s/it] [2024-05-22 00:31:13] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.1.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 31%|███ | 60/195 [01:01<02:47, 1.24s/it] 32%|███▏ | 62/195 [01:02<01:52, 1.18it/s] [2024-05-22 00:31:13] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.1.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 32%|███▏ | 62/195 [01:02<01:52, 1.18it/s] 32%|███▏ | 63/195 [01:02<01:31, 1.44it/s] [2024-05-22 00:31:13] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.10.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 32%|███▏ | 63/195 [01:02<01:31, 1.44it/s] [2024-05-22 00:31:13] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.10.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 32%|███▏ | 63/195 [01:02<01:31, 1.44it/s] 33%|███▎ | 65/195 [01:02<01:03, 2.04it/s] [2024-05-22 00:31:14] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.10.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 33%|███▎ | 65/195 [01:02<01:03, 2.04it/s] 34%|███▍ | 66/195 [01:03<01:08, 1.90it/s] [2024-05-22 00:31:14] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.10.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 34%|███▍ | 66/195 [01:03<01:08, 1.90it/s] [2024-05-22 00:31:14] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.10.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 34%|███▍ | 66/195 [01:03<01:08, 1.90it/s] 35%|███▍ | 68/195 [01:03<00:48, 2.61it/s] [2024-05-22 00:31:14] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.10.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 35%|███▍ | 68/195 [01:03<00:48, 2.61it/s] [2024-05-22 00:31:14] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.11.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 35%|███▍ | 68/195 [01:03<00:48, 2.61it/s] [2024-05-22 00:31:15] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.11.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 35%|███▍ | 68/195 [01:03<00:48, 2.61it/s] 36%|███▋ | 71/195 [01:03<00:32, 3.82it/s] [2024-05-22 00:31:15] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.11.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 36%|███▋ | 71/195 [01:03<00:32, 3.82it/s] 37%|███▋ | 72/195 [01:04<00:37, 3.31it/s] [2024-05-22 00:31:15] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.11.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 37%|███▋ | 72/195 [01:04<00:37, 3.31it/s] [2024-05-22 00:31:15] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.11.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 37%|███▋ | 72/195 [01:04<00:37, 3.31it/s] 38%|███▊ | 74/195 [01:04<00:30, 4.02it/s] [2024-05-22 00:31:16] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.11.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 38%|███▊ | 74/195 [01:04<00:30, 4.02it/s] [2024-05-22 00:31:16] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.12.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 38%|███▊ | 74/195 [01:04<00:30, 4.02it/s] [2024-05-22 00:31:16] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.12.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 38%|███▊ | 74/195 [01:04<00:30, 4.02it/s] 39%|███▉ | 77/195 [01:04<00:22, 5.26it/s] [2024-05-22 00:31:16] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.12.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 39%|███▉ | 77/195 [01:05<00:22, 5.26it/s] 40%|████ | 78/195 [01:05<00:27, 4.18it/s] [2024-05-22 00:31:16] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.12.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 40%|████ | 78/195 [01:05<00:27, 4.18it/s] [2024-05-22 00:31:16] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.12.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 40%|████ | 78/195 [01:05<00:27, 4.18it/s] 41%|████ | 80/195 [01:05<00:23, 4.83it/s] [2024-05-22 00:31:17] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.12.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 41%|████ | 80/195 [01:05<00:23, 4.83it/s] [2024-05-22 00:31:17] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.13.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 41%|████ | 80/195 [01:05<00:23, 4.83it/s] [2024-05-22 00:31:17] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.13.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 41%|████ | 80/195 [01:05<00:23, 4.83it/s] 43%|████▎ | 83/195 [01:06<00:18, 6.01it/s] [2024-05-22 00:31:17] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.13.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 43%|████▎ | 83/195 [01:06<00:18, 6.01it/s] 43%|████▎ | 84/195 [01:06<00:24, 4.57it/s] [2024-05-22 00:31:17] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.13.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 43%|████▎ | 84/195 [01:06<00:24, 4.57it/s] [2024-05-22 00:31:17] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.13.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 43%|████▎ | 84/195 [01:06<00:24, 4.57it/s] 44%|████▍ | 86/195 [01:06<00:20, 5.19it/s] [2024-05-22 00:31:18] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.13.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 44%|████▍ | 86/195 [01:06<00:20, 5.19it/s] [2024-05-22 00:31:18] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.14.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 44%|████▍ | 86/195 [01:06<00:20, 5.19it/s] [2024-05-22 00:31:18] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.14.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 44%|████▍ | 86/195 [01:06<00:20, 5.19it/s] 46%|████▌ | 89/195 [01:07<00:16, 6.37it/s] [2024-05-22 00:31:18] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.14.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 46%|████▌ | 89/195 [01:07<00:16, 6.37it/s] 46%|████▌ | 90/195 [01:07<00:21, 4.80it/s] [2024-05-22 00:31:18] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.14.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 46%|████▌ | 90/195 [01:07<00:21, 4.80it/s] [2024-05-22 00:31:19] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.14.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 46%|████▌ | 90/195 [01:07<00:21, 4.80it/s] 47%|████▋ | 92/195 [01:07<00:19, 5.41it/s] [2024-05-22 00:31:19] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.14.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 47%|████▋ | 92/195 [01:07<00:19, 5.41it/s] [2024-05-22 00:31:19] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.15.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 47%|████▋ | 92/195 [01:07<00:19, 5.41it/s] [2024-05-22 00:31:19] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.15.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 47%|████▋ | 92/195 [01:07<00:19, 5.41it/s] 49%|████▊ | 95/195 [01:08<00:15, 6.51it/s] [2024-05-22 00:31:19] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.15.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 49%|████▊ | 95/195 [01:08<00:15, 6.51it/s] 49%|████▉ | 96/195 [01:08<00:20, 4.81it/s] [2024-05-22 00:31:20] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.15.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 49%|████▉ | 96/195 [01:08<00:20, 4.81it/s] [2024-05-22 00:31:20] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.15.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 49%|████▉ | 96/195 [01:08<00:20, 4.81it/s] 50%|█████ | 98/195 [01:08<00:17, 5.41it/s] [2024-05-22 00:31:20] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.15.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 50%|█████ | 98/195 [01:08<00:17, 5.41it/s] [2024-05-22 00:31:20] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.16.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 50%|█████ | 98/195 [01:09<00:17, 5.41it/s] [2024-05-22 00:31:20] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.16.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 50%|█████ | 98/195 [01:09<00:17, 5.41it/s] 52%|█████▏ | 101/195 [01:09<00:14, 6.58it/s] [2024-05-22 00:31:20] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.16.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 52%|█████▏ | 101/195 [01:09<00:14, 6.58it/s] 52%|█████▏ | 102/195 [01:09<00:18, 4.92it/s] [2024-05-22 00:31:21] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.16.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 52%|█████▏ | 102/195 [01:09<00:18, 4.92it/s] [2024-05-22 00:31:21] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.16.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 52%|█████▏ | 102/195 [01:09<00:18, 4.92it/s] 53%|█████▎ | 104/195 [01:10<00:16, 5.44it/s] [2024-05-22 00:31:21] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.16.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 53%|█████▎ | 104/195 [01:10<00:16, 5.44it/s] [2024-05-22 00:31:21] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.17.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 53%|█████▎ | 104/195 [01:10<00:16, 5.44it/s] [2024-05-22 00:31:21] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.17.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 53%|█████▎ | 104/195 [01:10<00:16, 5.44it/s] 55%|█████▍ | 107/195 [01:10<00:13, 6.60it/s] [2024-05-22 00:31:21] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.17.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 55%|█████▍ | 107/195 [01:10<00:13, 6.60it/s] 55%|█████▌ | 108/195 [01:10<00:17, 4.94it/s] [2024-05-22 00:31:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.17.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 55%|█████▌ | 108/195 [01:10<00:17, 4.94it/s] [2024-05-22 00:31:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.17.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 55%|█████▌ | 108/195 [01:10<00:17, 4.94it/s] 56%|█████▋ | 110/195 [01:11<00:15, 5.47it/s] [2024-05-22 00:31:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.17.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 56%|█████▋ | 110/195 [01:11<00:15, 5.47it/s] [2024-05-22 00:31:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.18.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 56%|█████▋ | 110/195 [01:11<00:15, 5.47it/s] [2024-05-22 00:31:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.18.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 56%|█████▋ | 110/195 [01:11<00:15, 5.47it/s] 58%|█████▊ | 113/195 [01:11<00:12, 6.62it/s] [2024-05-22 00:31:22] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.18.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 58%|█████▊ | 113/195 [01:11<00:12, 6.62it/s] 58%|█████▊ | 114/195 [01:11<00:16, 4.95it/s] [2024-05-22 00:31:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.18.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 58%|█████▊ | 114/195 [01:11<00:16, 4.95it/s] [2024-05-22 00:31:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.18.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 58%|█████▊ | 114/195 [01:11<00:16, 4.95it/s] 59%|█████▉ | 116/195 [01:12<00:14, 5.55it/s] [2024-05-22 00:31:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.18.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 59%|█████▉ | 116/195 [01:12<00:14, 5.55it/s] [2024-05-22 00:31:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.19.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 59%|█████▉ | 116/195 [01:12<00:14, 5.55it/s] [2024-05-22 00:31:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.19.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 59%|█████▉ | 116/195 [01:12<00:14, 5.55it/s] 61%|██████ | 119/195 [01:12<00:11, 6.63it/s] [2024-05-22 00:31:23] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.19.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 61%|██████ | 119/195 [01:12<00:11, 6.63it/s] 62%|██████▏ | 120/195 [01:12<00:15, 4.78it/s] [2024-05-22 00:31:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.19.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 62%|██████▏ | 120/195 [01:12<00:15, 4.78it/s] [2024-05-22 00:31:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.19.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 62%|██████▏ | 120/195 [01:13<00:15, 4.78it/s] 63%|██████▎ | 122/195 [01:13<00:13, 5.33it/s] [2024-05-22 00:31:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.19.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 63%|██████▎ | 122/195 [01:13<00:13, 5.33it/s] [2024-05-22 00:31:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.2.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 63%|██████▎ | 122/195 [01:13<00:13, 5.33it/s] [2024-05-22 00:31:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.2.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 63%|██████▎ | 122/195 [01:13<00:13, 5.33it/s] 64%|██████▍ | 125/195 [01:13<00:10, 6.42it/s] [2024-05-22 00:31:24] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.2.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 64%|██████▍ | 125/195 [01:13<00:10, 6.42it/s] 65%|██████▍ | 126/195 [01:14<00:14, 4.77it/s] [2024-05-22 00:31:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.2.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 65%|██████▍ | 126/195 [01:14<00:14, 4.77it/s] [2024-05-22 00:31:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.2.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 65%|██████▍ | 126/195 [01:14<00:14, 4.77it/s] 66%|██████▌ | 128/195 [01:14<00:12, 5.32it/s] [2024-05-22 00:31:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.2.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 66%|██████▌ | 128/195 [01:14<00:12, 5.32it/s] [2024-05-22 00:31:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.20.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 66%|██████▌ | 128/195 [01:14<00:12, 5.32it/s] [2024-05-22 00:31:25] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.20.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 66%|██████▌ | 128/195 [01:14<00:12, 5.32it/s] 67%|██████▋ | 131/195 [01:14<00:09, 6.50it/s] [2024-05-22 00:31:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.20.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 67%|██████▋ | 131/195 [01:14<00:09, 6.50it/s] 68%|██████▊ | 132/195 [01:15<00:12, 4.89it/s] [2024-05-22 00:31:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.20.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 68%|██████▊ | 132/195 [01:15<00:12, 4.89it/s] [2024-05-22 00:31:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.20.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 68%|██████▊ | 132/195 [01:15<00:12, 4.89it/s] 69%|██████▊ | 134/195 [01:15<00:11, 5.50it/s] [2024-05-22 00:31:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.20.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 69%|██████▊ | 134/195 [01:15<00:11, 5.50it/s] [2024-05-22 00:31:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.21.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 69%|██████▊ | 134/195 [01:15<00:11, 5.50it/s] [2024-05-22 00:31:26] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.21.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 69%|██████▊ | 134/195 [01:15<00:11, 5.50it/s] 70%|███████ | 137/195 [01:15<00:08, 6.58it/s] [2024-05-22 00:31:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.21.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 70%|███████ | 137/195 [01:15<00:08, 6.58it/s] 71%|███████ | 138/195 [01:16<00:11, 4.80it/s] [2024-05-22 00:31:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.21.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 71%|███████ | 138/195 [01:16<00:11, 4.80it/s] [2024-05-22 00:31:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.21.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 71%|███████ | 138/195 [01:16<00:11, 4.80it/s] 72%|███████▏ | 140/195 [01:16<00:10, 5.42it/s] [2024-05-22 00:31:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.21.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 72%|███████▏ | 140/195 [01:16<00:10, 5.42it/s] [2024-05-22 00:31:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.22.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 72%|███████▏ | 140/195 [01:16<00:10, 5.42it/s] [2024-05-22 00:31:27] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.22.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 72%|███████▏ | 140/195 [01:16<00:10, 5.42it/s] 73%|███████▎ | 143/195 [01:16<00:07, 6.51it/s] [2024-05-22 00:31:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.22.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 73%|███████▎ | 143/195 [01:16<00:07, 6.51it/s] 74%|███████▍ | 144/195 [01:17<00:10, 4.87it/s] [2024-05-22 00:31:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.22.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 74%|███████▍ | 144/195 [01:17<00:10, 4.87it/s] [2024-05-22 00:31:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.22.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 74%|███████▍ | 144/195 [01:17<00:10, 4.87it/s] 75%|███████▍ | 146/195 [01:17<00:09, 5.41it/s] [2024-05-22 00:31:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.22.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 75%|███████▍ | 146/195 [01:17<00:09, 5.41it/s] [2024-05-22 00:31:28] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.23.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 75%|███████▍ | 146/195 [01:17<00:09, 5.41it/s] [2024-05-22 00:31:29] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.23.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 75%|███████▍ | 146/195 [01:17<00:09, 5.41it/s] 76%|███████▋ | 149/195 [01:17<00:06, 6.58it/s] [2024-05-22 00:31:29] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.23.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 76%|███████▋ | 149/195 [01:17<00:06, 6.58it/s] 77%|███████▋ | 150/195 [01:18<00:09, 4.82it/s] [2024-05-22 00:31:29] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.23.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 77%|███████▋ | 150/195 [01:18<00:09, 4.82it/s] [2024-05-22 00:31:29] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.23.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 77%|███████▋ | 150/195 [01:18<00:09, 4.82it/s] 78%|███████▊ | 152/195 [01:18<00:07, 5.44it/s] [2024-05-22 00:31:29] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.23.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 78%|███████▊ | 152/195 [01:18<00:07, 5.44it/s] [2024-05-22 00:31:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.3.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 78%|███████▊ | 152/195 [01:18<00:07, 5.44it/s] [2024-05-22 00:31:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.3.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 78%|███████▊ | 152/195 [01:18<00:07, 5.44it/s] 79%|███████▉ | 155/195 [01:18<00:06, 6.53it/s] [2024-05-22 00:31:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.3.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 79%|███████▉ | 155/195 [01:19<00:06, 6.53it/s] 80%|████████ | 156/195 [01:19<00:08, 4.72it/s] [2024-05-22 00:31:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.3.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 80%|████████ | 156/195 [01:19<00:08, 4.72it/s] [2024-05-22 00:31:30] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.3.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 80%|████████ | 156/195 [01:19<00:08, 4.72it/s] 81%|████████ | 158/195 [01:19<00:06, 5.33it/s] [2024-05-22 00:31:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.3.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 81%|████████ | 158/195 [01:19<00:06, 5.33it/s] [2024-05-22 00:31:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.4.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 81%|████████ | 158/195 [01:19<00:06, 5.33it/s] [2024-05-22 00:31:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.4.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 81%|████████ | 158/195 [01:19<00:06, 5.33it/s] 83%|████████▎ | 161/195 [01:20<00:05, 6.51it/s] [2024-05-22 00:31:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.4.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 83%|████████▎ | 161/195 [01:20<00:05, 6.51it/s] 83%|████████▎ | 162/195 [01:20<00:06, 4.83it/s] [2024-05-22 00:31:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.4.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 83%|████████▎ | 162/195 [01:20<00:06, 4.83it/s] [2024-05-22 00:31:31] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.4.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 83%|████████▎ | 162/195 [01:20<00:06, 4.83it/s] 84%|████████▍ | 164/195 [01:20<00:05, 5.43it/s] [2024-05-22 00:31:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.4.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 84%|████████▍ | 164/195 [01:20<00:05, 5.43it/s] [2024-05-22 00:31:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.5.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 84%|████████▍ | 164/195 [01:20<00:05, 5.43it/s] [2024-05-22 00:31:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.5.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 84%|████████▍ | 164/195 [01:20<00:05, 5.43it/s] 86%|████████▌ | 167/195 [01:21<00:04, 6.52it/s] [2024-05-22 00:31:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.5.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 86%|████████▌ | 167/195 [01:21<00:04, 6.52it/s] 86%|████████▌ | 168/195 [01:21<00:05, 4.74it/s] [2024-05-22 00:31:32] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.5.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 86%|████████▌ | 168/195 [01:21<00:05, 4.74it/s] [2024-05-22 00:31:33] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.5.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 86%|████████▌ | 168/195 [01:21<00:05, 4.74it/s] 87%|████████▋ | 170/195 [01:21<00:04, 5.35it/s] [2024-05-22 00:31:33] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.5.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 87%|████████▋ | 170/195 [01:21<00:04, 5.35it/s] [2024-05-22 00:31:33] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.6.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 87%|████████▋ | 170/195 [01:21<00:04, 5.35it/s] [2024-05-22 00:31:33] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.6.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 87%|████████▋ | 170/195 [01:22<00:04, 5.35it/s] 89%|████████▊ | 173/195 [01:22<00:03, 6.53it/s] [2024-05-22 00:31:33] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.6.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 89%|████████▊ | 173/195 [01:22<00:03, 6.53it/s] 89%|████████▉ | 174/195 [01:22<00:04, 4.84it/s] [2024-05-22 00:31:34] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.6.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 89%|████████▉ | 174/195 [01:22<00:04, 4.84it/s] [2024-05-22 00:31:34] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.6.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 89%|████████▉ | 174/195 [01:22<00:04, 4.84it/s] 90%|█████████ | 176/195 [01:22<00:03, 5.38it/s] [2024-05-22 00:31:34] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.6.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 90%|█████████ | 176/195 [01:22<00:03, 5.38it/s] [2024-05-22 00:31:34] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.7.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 90%|█████████ | 176/195 [01:23<00:03, 5.38it/s] [2024-05-22 00:31:34] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.7.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 90%|█████████ | 176/195 [01:23<00:03, 5.38it/s] 92%|█████████▏| 179/195 [01:23<00:02, 6.56it/s] [2024-05-22 00:31:34] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.7.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 92%|█████████▏| 179/195 [01:23<00:02, 6.56it/s] 92%|█████████▏| 180/195 [01:23<00:03, 4.82it/s] [2024-05-22 00:31:35] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.7.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 92%|█████████▏| 180/195 [01:23<00:03, 4.82it/s] [2024-05-22 00:31:35] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.7.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 92%|█████████▏| 180/195 [01:23<00:03, 4.82it/s] 93%|█████████▎| 182/195 [01:24<00:02, 5.43it/s] [2024-05-22 00:31:35] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.7.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 93%|█████████▎| 182/195 [01:24<00:02, 5.43it/s] [2024-05-22 00:31:35] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.8.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 93%|█████████▎| 182/195 [01:24<00:02, 5.43it/s] [2024-05-22 00:31:35] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.8.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 93%|█████████▎| 182/195 [01:24<00:02, 5.43it/s] 95%|█████████▍| 185/195 [01:24<00:01, 5.69it/s] [2024-05-22 00:31:35] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.8.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 95%|█████████▍| 185/195 [01:24<00:01, 5.69it/s] 95%|█████████▌| 186/195 [01:25<00:02, 4.42it/s] [2024-05-22 00:31:36] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.8.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 95%|█████████▌| 186/195 [01:25<00:02, 4.42it/s] [2024-05-22 00:31:36] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.8.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 95%|█████████▌| 186/195 [01:25<00:02, 4.42it/s] 96%|█████████▋| 188/195 [01:25<00:01, 5.07it/s] [2024-05-22 00:31:36] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.8.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 96%|█████████▋| 188/195 [01:25<00:01, 5.07it/s] [2024-05-22 00:31:36] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.9.input_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 96%|█████████▋| 188/195 [01:25<00:01, 5.07it/s] [2024-05-22 00:31:36] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.9.mlp.down_proj.weight[0m", shape: (4096, 11008), dtype: float16 | |
| 96%|█████████▋| 188/195 [01:25<00:01, 5.07it/s] 98%|█████████▊| 191/195 [01:25<00:00, 6.28it/s] [2024-05-22 00:31:37] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.9.mlp.gate_up_proj.weight[0m", shape: (22016, 4096), dtype: float16 | |
| 98%|█████████▊| 191/195 [01:25<00:00, 6.28it/s] 98%|█████████▊| 192/195 [01:26<00:00, 4.79it/s] [2024-05-22 00:31:37] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.9.post_attention_layernorm.weight[0m", shape: (4096,), dtype: float16 | |
| 98%|█████████▊| 192/195 [01:26<00:00, 4.79it/s] [2024-05-22 00:31:37] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.9.self_attn.qkv_proj.weight[0m", shape: (12288, 4096), dtype: float16 | |
| 98%|█████████▊| 192/195 [01:26<00:00, 4.79it/s] 99%|█████████▉| 194/195 [01:26<00:00, 5.34it/s] [2024-05-22 00:31:37] INFO huggingface_loader.py:174: [Not quantized] Parameter: "[1mmodel.layers.9.self_attn.o_proj.weight[0m", shape: (4096, 4096), dtype: float16 | |
| 99%|█████████▉| 194/195 [01:26<00:00, 5.34it/s] 100%|██████████| 195/195 [01:26<00:00, 2.26it/s] | |
| [2024-05-22 00:31:37] INFO huggingface_loader.py:196: Unloading HF weight file: /models/Llama-2-7b-chat-hf/pytorch_model-00001-of-00002.bin | |
| [2024-05-22 00:31:38] INFO stats.py:76: [92mTime usage[0m: HF loading: 44.115 sec; Pre-quantization mapping: 7.475 sec; Quantization: 0.000 sec | |
| [2024-05-22 00:31:38] INFO stats.py:90: [92mRAM usage[0m: Peak RAM: 9.291 GB. Total bytes loaded from disk: 12.551 GB | |
| [2024-05-22 00:31:38] INFO convert_weight.py:155: [92mParameter size[0m after quantization: 12.551 GB | |
| [2024-05-22 00:31:38] INFO convert_weight.py:160: [92mTotal parameters[0m: 6,738,415,616 | |
| [2024-05-22 00:31:38] INFO convert_weight.py:161: [92mBits per parameter[0m: 16.000 | |
| [2024-05-22 00:31:38] INFO convert_weight.py:166: Saved to directory: [1m/tmp/tmp8ew5r2yr[0m | |
| All finished, 131 total shards committed, record saved to /tmp/tmp8ew5r2yr/ndarray-cache.json | |